Re: Transaction APIs do not consult the DB router to choose DB connection

[Aymeric Augustin]

> On 2 Jun 2021, at 07:49, N Aditya  wrote:
> 
> Below are a few things I'd like to clarify:
> Are you referring to thread-locals as `global state/variables` in your 
> previous message ?
Yes.
> If so, why suggest something which you consider bad practise ?
You rejected the good practice before — atomic(using=...) so I'm looking for 
something that matches your constraint.
> As a consequence, if using thread-locals is considered bad software engg, 
> could you please give me a way by which I could pass on request metadata to 
> the router layer cleanly ?
Pass it explicitly with `atomic(using=...)`. I'm aware that it doesn't work for 
third-party libraries that you don't control. I don't have a perfect solution.
> The final recommendation you gave is something that isn't officially 
> supported and would fail when a threaded server or a fully async request path 
> is used.
> (Even your favourite search engine might not be of much help here)
Indeed, you'd need something slightly smarter, but since connections are 
thread-local in Django, it should be manageable.

I don't think you can meaningfully put "fully async request path" and "database 
transaction managed by the Django ORM" in the same sentence. You'd run the 
transaction in a thread pool, which brings you back to by previous case.

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/37A77A89-74F1-498D-AE41-9772528E4E32%40polytechnique.org.

Re: `Model.validate_unique` excluding partial unique constraint

[Gaga Ro]

Thanks for the thorough answer. I also realize now that it worked in my app 
only because of another side effect when my instance was saved..

I started to take a look at the ORM part where the check method should be 
implemented as I'm not used to it. Shouldn't .check() be implemented on Q 
and not on Expression? Or are you including Lookup / Q in it?

Then I'd guess it's just a matter of calling as_sql() from each part and 
assemble them. Everythings we need seems to be done in Query and we can't 
use it as it has to be linked to a model, so we would have to redo it? 
Although as_sql needs a compiler which itself needs a query. I admit I'm a 
bit lost in all those classes, everything seems to be too much linked to 
the models to do something without one.

If you have any more hints as to where I should look, thanks again.
Le mercredi 2 juin 2021 à 00:33:12 UTC+2, charettes a écrit :

> Hello there,
>
> Partial unique constraints are currently not supported during validation 
> for reasons described in this ticket[0].
>
> For example (inspired by this Github comment[1]), if you define the 
> following model
>
> class Article(models.Model):
> slug = models.CharField(max_length=100)
> deleted_at = models.DateTimeField(null=True)
>
> class Meta:
> constraints = [
> UniqueConstraint('slug', condition=Q(deleted_at=None), 
> name='unique_slug'),
> ]
>
> Then validate_unique must perform the following query to determine if the 
> constraint is violated
>
> SELECT NOT (%(deleted_at)s IS NULL) OR NOT EXISTS(SELECT 1 FROM article 
> WHERE NOT id = %(id)s AND slug = %(slug)s AND deleted_at IS NULL)
>
> In other words, the validation of a partial unique constraint must check 
> that either of these conditions are true
> 1. The provided instance doesn't match the condition
> 2. There's no existing rows matching the unique constraint (excluding the 
> current instance if it already exists)
>
> This is not something Django supports right now.
>
> In order to add proper support for this feature I believe (personal 
> opinion here feedback is welcome) we should follow these steps:
>
> 1. Add support for Expression.check(using: str) -> bool that would 
> translate IsNull(deleted_at, True).check('alias') into a backend 
> compatible 'SELECT %(deleted_at)s IS NULL' query and return whether or not 
> it passed. That would also allow the constructions of forms like
>
> (~Q(IsNull(deleted_at, True)) | 
> ~Exists(Article.objects.exclude(pk=pk).filter(slug=slug, 
> deleted_at=None)).check(using)
>
> 2. Add support for Constraint.validate(instance, excluded_fields) as 
> described in [0] that would build on top of Expression.check to implement 
> proper UniqueConstraint, CheckConstraint, and ExclusionConstraint 
> validation and allow for third-party app (e.g. django-rest-framework which 
> doesn't use model level validation[2]) to take advantage of this feature. 
> For example the unique_for_(date|month|year) feature of Date(Time)?Field 
> could be deprecated in favour of Constraint subclasses that implement 
> as_sql to enforce SQL level constraint if available by the current backend 
> and implement .validate to replace the special case logic we have currently 
> in place for these options[3].
>
> I hope this clarify the current situation.
>
> Cheers,
> Simon
>
> [0] https://code.djangoproject.com/ticket/30581#comment:7
> [1] https://github.com/django/django/pull/10796#discussion_r244216763
> [2] https://github.com/encode/django-rest-framework/issues/7173
> [3] 
> https://github.com/django/django/blob/e703b152c6148ddda1b072a4353e9a41dca87f90/django/db/models/base.py#L1062-L1084
>
> Le mardi 1 juin 2021 à 11:18:23 UTC-4, gaga...@gmail.com a écrit :
>
>> Hi,
>>
>> I changed several models from fields using `unique=True` to using 
>> `UniqueConstraint` with a condition in the Meta.
>>
>> As a side-effect, the uniqueness are no longer validated during cleaning 
>> of a Form and an integrity error is raised. This is because partial unique 
>> indexes are excluded :
>>
>> https://github.com/django/django/blob/e703b152c6148ddda1b072a4353e9a41dca87f90/django/db/models/options.py#L865-L874
>>
>> It seems that `total_unique_constraints` is also used to check for 
>> fields that should be unique (related fields and USERNAME_FIELD 
>> specifically).
>>
>> I tried modifying `total_unique_constraints` and the only tests which 
>> failed were related to the above concern and 
>> `test_total_ordering_optimization_meta_constraints` which also uses `
>> total_unique_constraints`. My application works fine and the validation 
>> error are correctly raised in my forms.
>>
>> The current behaviour of `Model.validate_unique` is also not the one I 
>> expected as my conditional `UniqueConstraint` were not used (which caused 
>> the integrity error).
>>
>> Am I missing something? Or should we use all constraints (including 
>> partial) in `Model.validate_unique`?
>>
>> If this is indeed what should be done, 

Re: Transaction APIs do not consult the DB router to choose DB connection

[Shai Berger]

Hi Aditya,

I think the basic issue is that the DB Routers framework is not the
right tool for the task you have in mind. You are trying to redirect
all database activity according to request parameters. The routers are
built for specific uses, and -- by design -- they don't cover all
cases; it's not just transactions. Off the top of my head, it's also raw
sql queries that you'd want to redirect in a similar way. So generally,
Aymeric's suggestion of changing the settings -- although I agree with
your criticism about its sensitivity to any form parallelism -- seems
closer to the right track than the idea of extending the routers.

I _think_ -- haven't looked in the details, so this may be a complete
blunder -- that you can achieve what you want using the Database
Instrumentation framework; it is built to allow you to intervene in the
execution of any SQL operation, at a lower level.

HTH,
Shai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20210602125405.04071a33.shai%40platonix.com.

Django security releases issued: 3.2.4, 3.1.12, and 2.2.24

[Carlton Gibson]

Details are available on the Django project weblog: 

https://www.djangoproject.com/weblog/2021/jun/02/security-releases/ 


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8712EB7A-190E-402D-A528-5CC2EE84CA91%40gmail.com.