Re: Question (potential feature request) on dropping database column in migration

[Todor Velichkov]

>
> Won’t generate migration file, which means database wise, nothing has 
> changed
>
You are the one generating migration files, Django is not forcing them.

The problem is not in Django, the problem is that this cannot be solved 
with a single deploy, there is not way for the old code to know whats 
coming in the future, which means you need to make a deploy just for this, 
and after that make a second deploy with the real changes. But if you are 
gonna make two deploys then you don't need Django to help you, you can 
solve this alone. Here is an example: Lets say we want to migrate from a 
full_name column to first_name and last_name columns. Here is what you can 
do:

1) Add first_name and last_name into the model and create a migration 
(which adds two columns and eventually populates them with data from 
full_name).
2) Change the code to make use of the new fields, remove full_name from the 
code, BUT do not make migration yet!
3) Deploy.
Now, during the deployment your old code will still use full_name which is 
still in the database, and when the new code gets live it will start using 
the new columns. 
Note: during the deployment you need to run migrations first, and reload 
the server after that, so your new code won't fail because new columns 
don't exist yet.
Note2: since you are running the migrations first, and restarting the 
server later, in theory its possible for a new data to appear into the 
database from the old code! Thus you wont have it migrated yet. e.g. A new 
user registered right after the migration ended and before the new code 
gets live, his name will be stored into the old full_name column! But you 
can fix this on the next step!
4) Now after the new code is live, and no one is using full_name anymore 
create a new migrations which will remove it from the database. In this 
migration you can double check that all data from full_name is properly 
migrated to the new columns.
5) Deploy again. (no code changes, just removing the old unused column from 
the database).



On Tuesday, October 2, 2018 at 3:47:10 AM UTC+3, martin_x wrote:
>
> Hi there,
>
> Thanks for making Django a great framework for starters :D
>
> Recently my team have run into problems when trying to remove a database 
> column from the model without stopping our Django server, below is the 
> timeline:
>
> 1. Before migration happens, we have column_A we want to remove, and 
> running on old release_1
> 2. Now we run migration to remove column_A, our webserver is still running 
> on old release_1 and serving requests
> 3. After migration, we ship the new release_2
>
> However, between step 2 and 3, there are requests coming and referencing 
> column_A we want to remove, and throws a exception of course.
>
> So is there anyway I can mark a database column a special state 
> (deprecated, purged in memory, etc), so it has the following affect:
> 1. Won’t generate migration file, which means database wise, nothing has 
> changed
> 2. However, when Django loads the model into memory, it will ignore 
> column_A completely. So query, create, update, etc won’t try to access 
> column_A.
>
> The reason we want to do it that way is so we can achieve no downtime, 
> seamless migration for our application. I believe more people will benefit 
> from this.
>
> Please let me know if more information is needed. Looking forward to 
> hearing from you.
>
> Martin
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a8bd0853-eb45-437d-ab87-f9586518e86f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

One more thing I forgot to add

I would add a clean_book method on the form and use it to check if the user 
> already have a book with that name.
>

You actually can't to this, because this just a simplified example where we 
have two fields only, what if we have a 3rd one and two of the fields are 
visible for the user? Then you will have to use Form.clean, where you will 
basically rewrite the unique check which Django decides to skip. 

On Monday, September 24, 2018 at 11:51:49 AM UTC+3, Todor Velichkov wrote:
>
> Then the form will raise a Validation error saying there is already a book 
>> with that user and that name. Confusing the user.
>>
> What error message are you gonna put instead? Book with that name already 
> exists? This is even more confusing, because its actually allowed to have 
> more books with that name.
> What about Hidden field errors? Aren't they confusing? 
>
> If you think a ModelForm giving guarantee that the save method will work 
>> make sense, the best way to proceed is a third party module.
>>
> Yes, lets keep pretending that ModelForms are just Forms, without Model in 
> their name. Server errors are much better than "Confusing" form errors.
>
>
> Simply do a Book.objects.filter(user=form.user, 
>> name=form.cleaned_data["name"]). exists()
>>
>  Why should I repeat something already implemented? More over, how are you 
> gonna keep this up to date with model unique constrains ? So every time I 
> touch unique_together I should go around all the forms and double check if 
> this wont break anything ?
>
>
> On Monday, September 24, 2018 at 9:09:58 AM UTC+3, ludovic coues wrote:
>>
>> First, that's a discussion for the Django user list, not the Django 
>> developers one.
>>
>> I would add a clean_book method on the form and use it to check if the 
>> user already have a book with that name. For that specific problems, that's 
>> the cleanest solution in my opinion. Simply do a 
>> Book.objects.filter(user=form.user, name=form.cleaned_data["name"]). 
>> exists() and if you get a hit, raise a Validation error. The error will be 
>> associated to the book field.
>>
>> If someone want to write a patch for adding full validation of a 
>> ModelForm, I wish them good luck. You could try a save commit=False but you 
>> will only get an integrity error and a string coming from the database. 
>> Which can be translated. You could check for your unique_together with an 
>> extra request. Then the form will raise a Validation error saying there is 
>> already a book with that user and that name. Confusing the user.
>>
>> If you think a ModelForm giving guarantee that the save method will work 
>> make sense, the best way to proceed is a third party module. Doing so let 
>> you do quick release for development and testing, user will be able try the 
>> module and see if it solves their problems. In my opinion, something 
>> generic won't help when validating unique_together relationship when one of 
>> the fields is not exposed to the user. But I could be wrong.
>>
>> On Sep 24, 2018 07:34, "Protik"  wrote:
>>
>> I am using Django 1.11. Further, adding a new book with user field 
>> disabled results in the following error:
>>
>> [image: Selection_046.png]
>>
>>
>> I have attached the code to reproduce the error.
>>
>>
>> On Monday, September 24, 2018 at 1:59:31 AM UTC+5:30, Todor Velichkov 
>> wrote:
>>>
>>> First thought: What is your Django version? The `disabled` attribute was 
>>> added in Django 1.9.
>>> However by looking at your code (w/o testing anything) after 
>>> `form.is_valid()` you should only call `form.save()`, no need to do 
>>> `commit=False` and manually assigning user, you have already done that in 
>>> the form constructor (by settings initial value + disabled attr).
>>>
>>> On Sunday, September 23, 2018 at 9:25:41 PM UTC+3, Protik wrote:
>>>>
>>>> Hi, Todor
>>>>
>>>> I have tested this solution and It looks like it works only when you 
>>>> don't disable the field (i.e the last line in the BookForm's `__init__()` 
>>>> method. My views looks like this:
>>>>
>>>>
>>>> def book_add(request):
>>>> user = get_object_or_404(User, id=1)
>>>>
>>>> if request.method == 'POST':
>>>>
>>>> f = BookForm(request.POST, user=user)
>>>> if f.is_valid():
>>>> book = f.save(commit=False)
>>>>   

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

>
> Then the form will raise a Validation error saying there is already a book 
> with that user and that name. Confusing the user.
>
What error message are you gonna put instead? Book with that name already 
exists? This is even more confusing, because its actually allowed to have 
more books with that name.
What about Hidden field errors? Aren't they confusing? 

If you think a ModelForm giving guarantee that the save method will work 
> make sense, the best way to proceed is a third party module.
>
Yes, lets keep pretending that ModelForms are just Forms, without Model in 
their name. Server errors are much better than "Confusing" form errors.


Simply do a Book.objects.filter(user=form.user, 
> name=form.cleaned_data["name"]). exists()
>
 Why should I repeat something already implemented? More over, how are you 
gonna keep this up to date with model unique constrains ? So every time I 
touch unique_together I should go around all the forms and double check if 
this wont break anything ?


On Monday, September 24, 2018 at 9:09:58 AM UTC+3, ludovic coues wrote:
>
> First, that's a discussion for the Django user list, not the Django 
> developers one.
>
> I would add a clean_book method on the form and use it to check if the 
> user already have a book with that name. For that specific problems, that's 
> the cleanest solution in my opinion. Simply do a 
> Book.objects.filter(user=form.user, name=form.cleaned_data["name"]). 
> exists() and if you get a hit, raise a Validation error. The error will be 
> associated to the book field.
>
> If someone want to write a patch for adding full validation of a 
> ModelForm, I wish them good luck. You could try a save commit=False but you 
> will only get an integrity error and a string coming from the database. 
> Which can be translated. You could check for your unique_together with an 
> extra request. Then the form will raise a Validation error saying there is 
> already a book with that user and that name. Confusing the user.
>
> If you think a ModelForm giving guarantee that the save method will work 
> make sense, the best way to proceed is a third party module. Doing so let 
> you do quick release for development and testing, user will be able try the 
> module and see if it solves their problems. In my opinion, something 
> generic won't help when validating unique_together relationship when one of 
> the fields is not exposed to the user. But I could be wrong.
>
> On Sep 24, 2018 07:34, "Protik" > wrote:
>
> I am using Django 1.11. Further, adding a new book with user field 
> disabled results in the following error:
>
> [image: Selection_046.png]
>
>
> I have attached the code to reproduce the error.
>
>
> On Monday, September 24, 2018 at 1:59:31 AM UTC+5:30, Todor Velichkov 
> wrote:
>>
>> First thought: What is your Django version? The `disabled` attribute was 
>> added in Django 1.9.
>> However by looking at your code (w/o testing anything) after 
>> `form.is_valid()` you should only call `form.save()`, no need to do 
>> `commit=False` and manually assigning user, you have already done that in 
>> the form constructor (by settings initial value + disabled attr).
>>
>> On Sunday, September 23, 2018 at 9:25:41 PM UTC+3, Protik wrote:
>>>
>>> Hi, Todor
>>>
>>> I have tested this solution and It looks like it works only when you 
>>> don't disable the field (i.e the last line in the BookForm's `__init__()` 
>>> method. My views looks like this:
>>>
>>>
>>> def book_add(request):
>>> user = get_object_or_404(User, id=1)
>>>
>>> if request.method == 'POST':
>>>
>>> f = BookForm(request.POST, user=user)
>>> if f.is_valid():
>>> book = f.save(commit=False)
>>> book.user = user
>>> book.save()
>>> messages.add_message(request, messages.INFO, 'book added.')
>>> return redirect('book_add')
>>> else:
>>> f = BookForm(user=user)
>>>
>>> return render(request, 'blog/book_add.html', {'f': f})
>>>
>>>
>>> def post_update(request, post_pk):
>>> user = get_object_or_404(User, id=1)
>>> book = get_object_or_404(Book, pk=post_pk)
>>> if request.method == 'POST':
>>> f = BookForm(request.POST, instance=book, user=user)
>>> if f.is_valid():
>>> post = f.save(commit=False)
>>> post.user = user
>>> post.save()
>>> messages.add_message(request, messages.

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

Protik, just use `self.fields['user'].initial = user.id` instead of just 
`user`. Just tested it and it work fine. 

On Monday, September 24, 2018 at 9:36:19 AM UTC+3, Protik wrote:
>
>
> Book.objects.filter(user=form.user, name=form.cleaned_data["name"]). 
> exists()
>
> This will only work for creating records, not updating.
>
> But what's the issue with the current solution posted by Todor. If you 
> look at the model form it looks like this:
>
> class BookForm(forms.ModelForm):
> class Meta:
>  model = Book
>  fields = ('user', 'name')
>
> def __init__(self, *args, **kwargs):
> user = kwargs.pop('user')
> super(BookForm, self).__init__(*args, **kwargs)
> self.fields['user'].widget = forms.HiddenInput()
> self.fields['user'].initial = user
> self.fields['user'].disabled = True
>
> Logically, It should work. But it is not, so far I have checked this code 
> with django version 1.11 and 2.1. 
>
> What do you think where the problem lies?
>
> On Monday, September 24, 2018 at 11:39:58 AM UTC+5:30, ludovic coues wrote:
>>
>> First, that's a discussion for the Django user list, not the Django 
>> developers one.
>>
>> I would add a clean_book method on the form and use it to check if the 
>> user already have a book with that name. For that specific problems, that's 
>> the cleanest solution in my opinion. Simply do a 
>> Book.objects.filter(user=form.user, name=form.cleaned_data["name"]). 
>> exists() and if you get a hit, raise a Validation error. The error will be 
>> associated to the book field.
>>
>> If someone want to write a patch for adding full validation of a 
>> ModelForm, I wish them good luck. You could try a save commit=False but you 
>> will only get an integrity error and a string coming from the database. 
>> Which can be translated. You could check for your unique_together with an 
>> extra request. Then the form will raise a Validation error saying there is 
>> already a book with that user and that name. Confusing the user.
>>
>> If you think a ModelForm giving guarantee that the save method will work 
>> make sense, the best way to proceed is a third party module. Doing so let 
>> you do quick release for development and testing, user will be able try the 
>> module and see if it solves their problems. In my opinion, something 
>> generic won't help when validating unique_together relationship when one of 
>> the fields is not exposed to the user. But I could be wrong.
>>
>> On Sep 24, 2018 07:34, "Protik"  wrote:
>>
>> I am using Django 1.11. Further, adding a new book with user field 
>> disabled results in the following error:
>>
>> [image: Selection_046.png]
>>
>>
>> I have attached the code to reproduce the error.
>>
>>
>> On Monday, September 24, 2018 at 1:59:31 AM UTC+5:30, Todor Velichkov 
>> wrote:
>>>
>>> First thought: What is your Django version? The `disabled` attribute was 
>>> added in Django 1.9.
>>> However by looking at your code (w/o testing anything) after 
>>> `form.is_valid()` you should only call `form.save()`, no need to do 
>>> `commit=False` and manually assigning user, you have already done that in 
>>> the form constructor (by settings initial value + disabled attr).
>>>
>>> On Sunday, September 23, 2018 at 9:25:41 PM UTC+3, Protik wrote:
>>>>
>>>> Hi, Todor
>>>>
>>>> I have tested this solution and It looks like it works only when you 
>>>> don't disable the field (i.e the last line in the BookForm's `__init__()` 
>>>> method. My views looks like this:
>>>>
>>>>
>>>> def book_add(request):
>>>> user = get_object_or_404(User, id=1)
>>>>
>>>> if request.method == 'POST':
>>>>
>>>> f = BookForm(request.POST, user=user)
>>>> if f.is_valid():
>>>> book = f.save(commit=False)
>>>> book.user = user
>>>> book.save()
>>>> messages.add_message(request, messages.INFO, 'book added.')
>>>> return redirect('book_add')
>>>> else:
>>>> f = BookForm(user=user)
>>>>
>>>> return render(request, 'blog/book_add.html', {'f': f})
>>>>
>>>>
>>>> def post_update(request, post_pk):
>>>> user = get_object_or_404(User, id=1)
>>>> book = get_object_or_404(B

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

First thought: What is your Django version? The `disabled` attribute was 
added in Django 1.9.
However by looking at your code (w/o testing anything) after 
`form.is_valid()` you should only call `form.save()`, no need to do 
`commit=False` and manually assigning user, you have already done that in 
the form constructor (by settings initial value + disabled attr).

On Sunday, September 23, 2018 at 9:25:41 PM UTC+3, Protik wrote:
>
> Hi, Todor
>
> I have tested this solution and It looks like it works only when you don't 
> disable the field (i.e the last line in the BookForm's `__init__()` method. 
> My views looks like this:
>
>
> def book_add(request):
> user = get_object_or_404(User, id=1)
>
> if request.method == 'POST':
>
> f = BookForm(request.POST, user=user)
> if f.is_valid():
> book = f.save(commit=False)
> book.user = user
> book.save()
> messages.add_message(request, messages.INFO, 'book added.')
> return redirect('book_add')
> else:
> f = BookForm(user=user)
>
> return render(request, 'blog/book_add.html', {'f': f})
>
>
> def post_update(request, post_pk):
> user = get_object_or_404(User, id=1)
> book = get_object_or_404(Book, pk=post_pk)
> if request.method == 'POST':
> f = BookForm(request.POST, instance=book, user=user)
> if f.is_valid():
> post = f.save(commit=False)
> post.user = user
> post.save()
> messages.add_message(request, messages.INFO, 'Post added.')
> return redirect('post_update', post.pk)
> else:
> f = BookForm(instance=book, user=user)
>
> return render(request, 'blog/book_update.html', {'f': f})
>
>
> The code for models and modelform is exactly same as yours.
>
>
> Am I doing something wrong?
>
>
> On Sunday, September 23, 2018 at 9:11:55 PM UTC+5:30, Todor Velichkov 
> wrote:
>>
>> You can use the `disabled 
>> <https://docs.djangoproject.com/en/1.11/ref/forms/fields/#disabled>` 
>> attribute on form fields with a combination of HiddenInput 
>> <https://docs.djangoproject.com/en/2.1/ref/forms/widgets/#hiddeninput>
>>
>> Using the Book example from the first comment it will look like this:
>>   
>> class BookForm(forms.ModelForm):
>> class Meta:
>> model = Book
>> fields = ('user', 'name')
>> 
>> def __init__(self, *args, **kwargs):
>> user = kwargs.pop('user')
>> super(BookForm, self).__init__(*args, **kwargs)
>> self.fields['user'].widget = forms.HiddenInput()
>> self.fields['user'].initial = user
>> self.fields['user'].disabled = True
>>
>>
>> First we hide the the user field because we dont want the user to see it, 
>> and at the same time keeping it inside form fields we wont prevent the 
>> unique_together validation.
>> Second - we disable the field and programmatically set initial value to 
>> be used during form validation
>>
>> On Sunday, September 23, 2018 at 4:57:15 PM UTC+3, Protik wrote:
>>>
>>> Hi  Todor
>>>
>>> I am experiencing the same problem. Can you please post the 
>>> possible solution?
>>>
>>> On Tuesday, October 10, 2017 at 8:26:32 AM UTC+5:30, Todor Velichkov 
>>> wrote:
>>>>
>>>> It does? Can you give me a link to that please?
>>>>>
>>>>
>>>> Pard me, it does not explicitly say to set values programmatically, but 
>>>> that this is the place to go when fields depend on each other, and since 
>>>> clean is a multi-step process which does not include only field 
>>>> validation, 
>>>> but settings values too, it kind of makes sense.
>>>>
>>>> 1) Form and field validation  
>>>> <https://docs.djangoproject.com/en/1.11/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other>
>>>>
>>>> The form subclass’s clean() method can perform validation that 
>>>>> requires access to multiple form fields. This is where you might put in 
>>>>> checks such as “if field A is supplied, field B must contain a valid 
>>>>> email address”. *This method can return a completely different 
>>>>> dictionary if it wishes, which will be used as the cleaned_data*. 
>>>>>
>>>>
>>>> 2) Cleaning and validating fields that depend on each other 
>>>> <https://docs.djangoproject.com/en/1.11/ref/forms/va

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

You can use the `disabled 
<https://docs.djangoproject.com/en/1.11/ref/forms/fields/#disabled>` 
attribute on form fields with a combination of HiddenInput 
<https://docs.djangoproject.com/en/2.1/ref/forms/widgets/#hiddeninput>

Using the Book example from the first comment it will look like this:
  
class BookForm(forms.ModelForm):
class Meta:
model = Book
fields = ('user', 'name')

def __init__(self, *args, **kwargs):
user = kwargs.pop('user')
super(BookForm, self).__init__(*args, **kwargs)
self.fields['user'].widget = forms.HiddenInput()
self.fields['user'].initial = user
self.fields['user'].disabled = True


First we hide the the user field because we dont want the user to see it, 
and at the same time keeping it inside form fields we wont prevent the 
unique_together validation.
Second - we disable the field and programmatically set initial value to be 
used during form validation

On Sunday, September 23, 2018 at 4:57:15 PM UTC+3, Protik wrote:
>
> Hi  Todor
>
> I am experiencing the same problem. Can you please post the 
> possible solution?
>
> On Tuesday, October 10, 2017 at 8:26:32 AM UTC+5:30, Todor Velichkov wrote:
>>
>> It does? Can you give me a link to that please?
>>>
>>
>> Pard me, it does not explicitly say to set values programmatically, but 
>> that this is the place to go when fields depend on each other, and since 
>> clean is a multi-step process which does not include only field validation, 
>> but settings values too, it kind of makes sense.
>>
>> 1) Form and field validation  
>> <https://docs.djangoproject.com/en/1.11/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other>
>>
>> The form subclass’s clean() method can perform validation that requires 
>>> access to multiple form fields. This is where you might put in checks such 
>>> as “if field A is supplied, field B must contain a valid email 
>>> address”. *This method can return a completely different dictionary if 
>>> it wishes, which will be used as the cleaned_data*. 
>>>
>>
>> 2) Cleaning and validating fields that depend on each other 
>> <https://docs.djangoproject.com/en/1.11/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other>
>>
>> Suppose we add another requirement to our contact form: if the cc_myself 
>>> field is True, the subject must contain the word "help". *We are 
>>> performing validation on more than one field at a time, so the form’s 
>>> clean() method is a good spot to do this.*
>>>
>>
>> 3) Model.clean() 
>> <https://docs.djangoproject.com/en/1.11/ref/models/instances/#django.db.models.Model.clean>
>>
>> This method should be used to provide custom model validation, *and to 
>>> modify attributes on your model if desired*. For instance, you could 
>>> use it to automatically provide a value for a field, or to do validation 
>>> that requires access to more than a single field.
>>>
>>
>> Please, don't get me wrong, I'm far away from the idea of deprecating 
>> `commit=False`. I just personally try to void it and trying to explain my 
>> arguments. 
>>
>> The generic error message is something that I supply in some forms of 
>>> mine where race conditions can happen due to high concurrency. This is why 
>>> I guard form.save, catch database errors and then use form.add_error to add 
>>> a generic error message asking for a retry.
>>>
>>
>> Yes, that's definitely the place when something outside the validation 
>> process can happen, but you don't need `commit=False` here ;) 
>>
>> One alternative approach to validate the instance w/o `commit=False` 
>> right now would be to change the form like so:
>>
>> class BookForm(forms.ModelForm):
>>
>> class Meta:
>> model = Book
>> fields = ('name', 'user')
>> widgets = {'user': forms.HiddenInput()}
>>
>> def __init__(self, *args, **kwargs):
>> super(BookForm, self).__init__(*args, **kwargs)
>> self.fields['user'].disabled = True
>>
>> #in the view
>> form = BookForm(data={'name': 'Harry Potter'}, initial={'user': user})
>>
>> But if we compare this with the form in the first post, we are just 
>> Fixing/Patching it, i.e. fighting with it in order to make it work for us.
>>
>> However, there is one more little difference here, which I want to point 
>> out. Image we didn't had the unique constrain, and we just wanted to hide 
>> the user field 

Re: QueryDict and ordering

[Todor Velichkov]

Is this what you are looking for?


from django.http import QueryDict
from collections import OrderedDict

class OrderedQueryDict(QueryDict, OrderedDict):
pass

my_dict = OrderedQueryDict(request.META['QUERY_STRING'])

print my_dict.items()



On Friday, January 5, 2018 at 5:07:41 PM UTC+2, Ole Laursen wrote:
>
> Hi!
>
> Would it be possible to derive QueryDict (i.e. MultiValueDict) from an 
> OrderedDict instead of dict?
>
> I'm finding it increasingly irritating that the original order is kept by 
> the whole stack right until Django puts it into a dict. It makes some 
> highly dynamic form situations more tedious to handle.
>
> Now that Python 3.6 preserves the order of dicts as an implementation 
> detail, there should not be any performance overhead as far as I'm aware.
>
>
> Ole
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1913a9a7-e2e2-4373-8486-7ffd9bbf53fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

>
> It does? Can you give me a link to that please?
>

Pard me, it does not explicitly say to set values programmatically, but 
that this is the place to go when fields depend on each other, and since 
clean is a multi-step process which does not include only field validation, 
but settings values too, it kind of makes sense.

1) Form and field validation  
<https://docs.djangoproject.com/en/1.11/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other>

The form subclass’s clean() method can perform validation that requires 
> access to multiple form fields. This is where you might put in checks such 
> as “if field A is supplied, field B must contain a valid email address”. 
> *This 
> method can return a completely different dictionary if it wishes, which 
> will be used as the cleaned_data*. 
>

2) Cleaning and validating fields that depend on each other 
<https://docs.djangoproject.com/en/1.11/ref/forms/validation/#cleaning-and-validating-fields-that-depend-on-each-other>

Suppose we add another requirement to our contact form: if the cc_myself 
> field is True, the subject must contain the word "help". *We are 
> performing validation on more than one field at a time, so the form’s 
> clean() method is a good spot to do this.*
>

3) Model.clean() 
<https://docs.djangoproject.com/en/1.11/ref/models/instances/#django.db.models.Model.clean>

This method should be used to provide custom model validation, *and to 
> modify attributes on your model if desired*. For instance, you could use 
> it to automatically provide a value for a field, or to do validation that 
> requires access to more than a single field.
>

Please, don't get me wrong, I'm far away from the idea of deprecating 
`commit=False`. I just personally try to void it and trying to explain my 
arguments. 

The generic error message is something that I supply in some forms of mine 
> where race conditions can happen due to high concurrency. This is why I 
> guard form.save, catch database errors and then use form.add_error to add a 
> generic error message asking for a retry.
>

Yes, that's definitely the place when something outside the validation 
process can happen, but you don't need `commit=False` here ;) 

One alternative approach to validate the instance w/o `commit=False` right 
now would be to change the form like so:

class BookForm(forms.ModelForm):

class Meta:
model = Book
fields = ('name', 'user')
widgets = {'user': forms.HiddenInput()}

def __init__(self, *args, **kwargs):
super(BookForm, self).__init__(*args, **kwargs)
self.fields['user'].disabled = True

#in the view
form = BookForm(data={'name': 'Harry Potter'}, initial={'user': user})

But if we compare this with the form in the first post, we are just 
Fixing/Patching it, i.e. fighting with it in order to make it work for us.

However, there is one more little difference here, which I want to point 
out. Image we didn't had the unique constrain, and we just wanted to hide 
the user field from the form, and we do it this way (instead of the one 
with the `instance` kwarg as in the first post). Doing this and adding the 
unique_constrain later on would yield to no more core changes (except 
changing the error msg if too ambiguous). While using the `commit=False` 
approach would require further code changes and it would be multiplied by 
the number of views the form is being used, and by the number of forms 
where the field has been omitted (i.e. commit=False has been used). Its a 
slight difference, but can lead to a big wins.

About the error message, to be honest I don't know, gonna keep thinking 
about it, one thing that came to mind is to strip the missing fields, i.e. 
instead of "Book with this User and Name already exists." to become: "Book 
with this Name already exists." but it really depends on the context. The 
one thing that I know for sure is that personally I would definitely prefer 
an ambiguous message, rather than a 500 server error.



On Monday, October 9, 2017 at 10:52:50 AM UTC+3, Florian Apolloner wrote:
>
> Hi,
>
> On Monday, October 9, 2017 at 8:52:53 AM UTC+2, Todor Velichkov wrote:
>>
>> Settings values programmatically is a cumulative operation most of the 
>> time, however when its not and things depend on each other (like your 
>> example), then even the docs suggests than one can use the form.clean 
>> method. 
>>
>
> It does? Can you give me a link to that please?
>
> If there is some other dependency outside form.cleaned_data I would prefer 
>> to use dependency injection in order to get this data and do the job done. 
>> I'm sorry I just see commit=False as an anti-pattern, because the 
>> validation needs to be repeated after that (as your example in the first 
>> post).
>

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

Settings values programmatically is a cumulative operation most of the 
time, however when its not and things depend on each other (like your 
example), then even the docs suggests than one can use the form.clean 
method. If there is some other dependency outside form.cleaned_data I would 
prefer to use dependency injection in order to get this data and do the job 
done. I'm sorry I just see commit=False as an anti-pattern, because the 
validation needs to be repeated after that (as your example in the first 
post).

Showing an error message which the user cannot correct is a horrible UX 
indeed, but still its a UX, and some people may find it as a better 
alternative to a `500 server error page`, where there is no UX at all. Even 
a generic message like 'sorry we messed up' would be useful, because the 
user data that will be preserved into the form. However, in the example 
shown here, this is not even the case, there is something that the user can 
do to prevent the error.

Finally, yes, there may never be a 100% guarantee that an instance can 
actually be saved, but this would be outside of Django's scope? In terms of 
a model.save() things are deterministic and we should do doing our best to 
ensure that the operation is valid for execution, right? In the example I'm 
showing is not something outside of Django's knowledge to make it 
unexpected/unpreventable.

Thank you,
Todor.


On Sunday, October 8, 2017 at 7:36:58 PM UTC+3, Florian Apolloner wrote:
>
>
>
> On Saturday, October 7, 2017 at 5:07:31 PM UTC+2, Todor Velichkov wrote:
>>
>> I believe this could save a lot of headache to people. If there is no 
>> guarantee that an instance cannot be saved after the form has been 
>> validated, then do not give me an option to shoot myself into the foot.
>>
>
> Even if the whole instance were validated there is __never__ and can 
> __never__ be any guarantee that the instance can actually be saved.
>
>
>> My only concern here actually is not that we didn't know what will happen 
>> (I think we know that), but how to show the user that the developer got 
>> messed up, i.e. how to render an error for a field that is not rendered to 
>> the user? Should this error be propagated to a NON-FIELD error or something 
>> else?
>>
>
> Showing an error message which the user cannot correct because the field 
> is not in the form is horrible UX.
>
> I feel its perfectly fine to ask for a complete model instance at this 
>> stage of the validation process (lots of methods got passed from the 
>> beginning - form.clean() and instance.clean()).
>>
>
> I strongly disagree, especially for values which are set programmatically 
> after validation (with the use of save(commit=False)) depending on values 
> the user chose. Ie when the user chooses a category for a bug report and 
> the code then automatically assigns another user as owner of that 
> reuqest/ticket.
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/61db7623-a32b-4d1d-890d-f1ce463d8c91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

Thank you for the replay Florian, I will think about what you said that 
Django shouldn't make any assumptions about fields outside the form, but my 
intuition 
tells me that there are more staff which we could be retrieved from the 
ModelForm's.

Some of my initial thoughts:

If the ModelForm doesn't care about anything outside the defined fields, 
then this is just a regular Form, the only difference is that it can use 
Models as a short-cut to be defined. (ModelFormFactory?)
If we agree on this (I'm perfectly fine to agree that this could be the 
scope of a ModelForm), then I would probably not expect to see a method 
like `form.save()`, just force me to use `form.cleaned_data` to build my 
instance, make it more clear that the ModelForm is more like a FormFactory 
class instead of throwing me options like `commit=False`. I believe this 
could save a lot of headache to people. If there is no guarantee that an 
instance cannot be saved after the form has been validated, then do not 
give me an option to shoot myself into the foot.

However, why I could expect the ModelForm to be something more...

This is a ModelForm, yes it may sit into the `forms` namespace, but still 
its half `Model` and a half `Form`, pretending to be a form-only class is 
pity for something that do so much heavy-lifting to remove all the tedious 
tasks from us (love it for this). Use all the metadata we have into the 
model to validate the instance. My only concern here actually is not that 
we didn't know what will happen (I think we know that), but how to show the 
user that the developer got messed up, i.e. how to render an error for a 
field that is not rendered to the user? Should this error be propagated to 
a NON-FIELD error or something else? I don't know yet, maybe no, maybe this 
field should not be validated and lead to a server error. But, however, in 
the scope of the problem shown here (a multi-field-level unique constrain) 
this is not an issue, because this is not a field-level error, this is a 
multi-field-unique constrains, so maybe the only thing that needs to be 
changes, is to stop respecting the excluded fields when validating 
multi-field level unique constrains? I feel its perfectly fine to ask for a 
complete model instance at this stage of the validation process (lots of 
methods got passed from the beginning - form.clean() and instance.clean()).

I would love to see more opinions on the topic. Thank you.

On Saturday, October 7, 2017 at 1:55:24 PM UTC+3, Florian Apolloner wrote:
>
> I think the current behaviour is correct. Django shouldn't make any 
> assumptions about fields not in the form since it doesn't know what will 
> happen… In that sense I disagree with the statement:
>
> > From my point of view when I define a ModelForm I imagine that I have to 
> only explain what the user needs to see when the form gets rendered.
>
> To get what you want use something along the lines of: 
> https://gist.github.com/apollo13/ef72035c5f864174ad3c968dec60d88a
>
> Cheers,
> Florian
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/99469dcc-5850-4168-b490-2ca7accf8ebc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

I will best explain what I mean with an example. 

Lets say we have a simple model called Book.

class Book(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=255)

class Meta:
unique_together = ('user', 'name')

so `Books` are linked to users, and we have added an unique constrain 
because we don't want a user to have the same book twice. 

Now lets say we want to allow users to create/update books, when we do this 
we would only need their book.name, we already knew who they are in the 
system.
So a simple ModelForm would look like this:

class BookForm(forms.ModelForm):
class Meta:
 model = Book
 fields = ('name',)

Now since i've omitted the `user` field and I can't have a book w/o a user, 
its my responsibility to populate it. Thus when I create new form i will 
always a pass an instance with a user.
But before we do this, lets manually create 1 book for this user.

user = User.objects.get(pk=1)
book1 = Book.objects.create(user=user, name='Harry Potter')

#now lets build our form
book2 = Book(user=user)
form = BookForm(data={'name': 'Harry Potter'}, instance=book2)

if form.is_valid(): #yes the form will be valid
book2 = form.save() #IntegrityError will be raised with UNIQUE 
constraint failed

Now this happens because `ModelForm.validate_unique 
` 
collects a set of fields to pass as an exclude fields into 
`instance.validate_unique`. 

Please, can someone explain me why is this done in such a manner, and what 
will be the correct way approach this problem.

>From my point of view when I define a ModelForm I imagine that I have to 
only explain what the user needs to see when the form gets rendered.
But when the instance is being validated, I would always want to validate 
this instance as a whole (with the fields that I've omitted).



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e5e49072-9d33-4faf-99f7-f7339437c73a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using EXISTS instead of IN for subqueries

[Todor Velichkov]

How does this query look like? Not sure if this is gonna help, but you can 
take a look at Exists() subqueries 


On Wednesday, September 6, 2017 at 4:29:21 PM UTC+3, Stephan Seyboth wrote:
>
> Sorry for resurrecting this ancient thread, but it was the last direct 
> mention I could find about this topic.
>
> Curious to hear what happened to Anssi's proposal to change the ORM to use 
> EXISTS subqueries instead of IN subqueries. Looks like the ORM still uses 
> IN as of Django 1.11.4.
>
> One of our queries just started running over 1000x slower from one day to 
> the next b/c of the issues Postgres has with IN.
>
> Are there any suggested workarounds that don't require resorting to raw 
> SQL?
>
> On Tuesday, March 26, 2013 at 5:40:35 PM UTC+1, Anssi Kääriäinen wrote:
>>
>> On Tuesday, March 26, 2013 5:19:52 PM UTC+2, Michael Manfre wrote:
>>>
>>>
>>>
>>> On Tue, Mar 26, 2013 at 10:23 AM, Tim Chase >> > wrote:
>>>
  EXISTS also has some nice features
 like the ability to do testing against multiple columns, i.e., you
 can't do something like

   select *
   from tbl1
   where (col1, col2) in (
select a, b
from tbl2
)

 but that's a simple EXISTS query.

>>>
>>> Agreed, EXISTS is more flexible and at least to me, often easier to 
>>> read, format, and maintain. Assuming this is implemented with a database 
>>> feature, I'll most likely enable the EXISTS change for django-mssql. My 
>>> main objections to a blanket change is to not have the specific behaviors 
>>> of one database dictate how every other database must behave. Some one 
>>> needs to be the voice of 3rd party database backends and it appears for the 
>>> time being, I'm wearing that hat.
>>>
>>
>> One of the main reasons for the change is that EXISTS allows for queries 
>> that are impossible with IN. In addition EXISTS semantics regarding NULLs 
>> is wanted. And, at least PostgreSQL performs better with NOT EXISTS than 
>> NOT IN. Granted, the better performance on PostgreSQL is perhaps the most 
>> important reason for me, but this change is not only about that.
>>
>> The main reason for this thread was to find out if there are some 
>> databases where performance of EXISTS is worse than IN. The DB feature 
>> approach seems good because it allows investigating performance 
>> characteristics one DB at time.
>>
>>  
>>>
 > > MSSQL is a 2nd-class citizen in the Django world, so I'm +1
 >
 > Reasoning like that helps to keep it in its place.

 MSSQL's lack of certain core features is what does that.
 OFFSET/LIMIT provided by pretty much every other DB vendor?  Use
 baroque hacks to get the same functionality.  
>>>
>>>
>>> The non "baroque hack" way of doing limit offset is with nested selects. 
>>> Maybe someday the non-standard LIMIT/OFFSET keywords will get added to the 
>>> standard (I truly hope this happens) so Oracle, MSSQL, DB2, and Informix 
>>> could share SQL with postgres and mysql without needing to mangle it.
>>>
>>
>> Hmmh, this means Oracle, MSSQL, DB2 and Informix are doing more or less 
>> the same thing for limit/offset support? If so, then having a more generic 
>> approach to this problem than having a custom compiler per backend might be 
>> worth it...
>>
>> BTW there is already something like LIMIT and OFFSET in SQL 2008 
>> standard. The syntax is different than LIMIT/OFFSET, and supported only by 
>> some vendors... See 
>> https://en.wikipedia.org/wiki/Select_%28SQL%29#FETCH_FIRST_clause
>>
>>  - Anssi
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/43e19ef1-5a07-4608-94f4-82ba10ad9a85%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Automatic prefetching in querysets

[Todor Velichkov]

+1 to just add the queryset method it gives a fine granular level of 
control. Moreover when not used we could display warnings which can help 
people detect these O(n) query cases as Tom Forbes initially suggested.

On Friday, August 18, 2017 at 9:08:11 AM UTC+3, Anssi Kääriäinen wrote:
>
> On Thursday, August 17, 2017 at 11:30:45 PM UTC+3, Aymeric Augustin wrote:
>>
>> Hello, 
>>
>> > On 17 Aug 2017, at 14:48, Luke Plant  wrote: 
>> > 
>> > On 16/08/17 23:17, Aymeric Augustin wrote: 
>> >> It should kick in only when no select_related or prefetch_related is 
>> in effect, to avoid interfering with pre-existing optimizations. It's still 
>> easy to construct an example where it would degrade performance but I don't 
>> think such situations will be common in practice. 
>> > 
>> > I think Ansii's example is a non-contrived and potentially very common 
>> one where we will make things worse if this is default behaviour, 
>> especially for the non-expert programmer this behaviour is supposed to 
>> help: 
>> > 
>> > 
>> > if qs: 
>> > qs[0].do_something() 
>> > 
>> > (where do_something() access relations, or multiple relations). We will 
>> end up with *multiple* large unnecessary queries instead of just one. The 
>> difference is that the first one is easy to explain, the remainder are much 
>> harder, and will contribute even more to the "Django is slow/ORMs are bad" 
>> feeling. 
>>
>>
>> I wasn't convinced by this example because it's already sub-optimal 
>> currently. `if qs` will fetch the whole queryset just to check if it has at 
>> least one element. 
>>
>> Assuming there's 1000 elements in qs, if you don't care about a 1000x 
>> inefficiency, I don't expect you to care much about two or three 1000x 
>> inefficiencies...
>>
>
> I agree that this example isn't particularly worrying. It's something an 
> experienced developer wouldn't do. On the other hand, we are aiming at 
> making things simpler for non-experienced developers.
>
> To me the worrying part here is that we really don't have any data or 
> experience about if the cure will be worse than the disease. Likely not, 
> but what do we gain by taking risks here?
>
> Maybe we should just add the queryset method. This is the smallest atomic 
> task that can be done. Even if there's only the queryset method available, 
> it's possible to enable prefetches per model by using a Manager.
>
>  - Anssi
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d956a26d-adcf-443f-80c8-7b7ba290d2b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding signals to bulk update/create operations

[Todor Velichkov]

@Tim, sorry about that, I did a search before I posted, but it looks like 
it slipped away somehow.

@Anssi, thank you for your response.
I really haven't think about threat safety, but maybe its because in our 
specific usage it's close to impossible to happen.

What do you think about this:
1) Put everything into a transaction
2) Before update, count the queryset.
3) Fire the pre_update with the queryset, and the count.
4) Do the actual update -> get the updated rows
5) Assert the updated rows is equal to the queryset.count().
6) Revert the transaction if there is a difference.

No pk fetching, this is left to the be implemented by the listener if he 
needs it.

On Friday, March 31, 2017 at 8:01:45 AM UTC+3, Anssi Kääriäinen wrote:
>
> The problem with passing the queryset is that it's possible that some 
> object is added to or removed from the queryset between the pre_update and 
> actual update execution. To avoid this the execution should go somewhere 
> along the lines of:
>1) if there is pre_update or post_update do stages 2-5, if not, update 
> as with current code
>2) fetch primary keys for models to be updated to a set, with 
> .select_for_update() applied
>3) fire pre_update, give the primary key set as argument
>4) do the update against a queryset with .filter(pk__in=pk_set)
>5) fire post_update with primary key set as argument
>
> This way the pre and post update signals will execute against a fixed set 
> of instances. The bad part is that this can significantly slow down the 
> .update() call, but to get actually safe signals, I don't see a way around 
> that.
>
>  - Anssi
>
> On Friday, March 31, 2017 at 3:51:34 AM UTC+3, Tim Graham wrote:
>>
>> There's an accepted ticket about adding pre_update and post_update 
>> signals: https://code.djangoproject.com/ticket/21461. From a quick 
>> glance, I think this is what you're proposing.
>>
>> On Thursday, March 30, 2017 at 4:28:00 PM UTC-4, Todor Velichkov wrote:
>>>
>>> Consider the following piece of code:
>>>
>>> @receiver(pre_save, sender=MyModel)
>>> def my_handler(sender, **kwargs):
>>> instance = kwargs['instance']
>>> if instance.verified:
>>> do_something(instance)
>>> else:
>>> do_else(instance)
>>>
>>>
>>>
>>> Its good, because it keeps `MyModel` decoupled from `do_something`
>>>
>>> But there is one flaw. If we do:
>>>
>>> MyModel.objects.filter(verified=False).update(verified=True)
>>>
>>> we are screwed, `do_something` is not executed, and our models get 
>>> out-of-sync.
>>>
>>> If we try to get smart and manually fire the pre_save signal for each 
>>> instance, we are gonna have a hard time.
>>> Its gonna be slow.
>>> And its gonna be memory inefficient.
>>>
>>> We already experienced it in our app.
>>>
>>> So our new approach is like this:
>>>
>>> pre_bulk_update = Signal(providing_args=["queryset", "update_kwargs"
>>> ])
>>> post_bulk_update = Signal(providing_args=["update_kwargs",])
>>>
>>> pre_bulk_create = Signal(providing_args=["objs", "batch_size"])
>>> post_bulk_create = Signal(providing_args=["objs", "batch_size"])
>>>
>>>
>>> class MyModelQuerySet(models.QuerySet):
>>> def update(self, **kwargs):
>>> pre_bulk_update.send(sender=self.model, queryset=self, 
>>> update_kwargs=kwargs)
>>> res = super(MyModelQuerySet, self).update(**kwargs)
>>> # The queryset will be altered after the update call
>>> # so no reason to send it.
>>> post_bulk_update.send(sender=self.model, update_kwargs=
>>> kwargs)
>>> return res
>>>
>>> def bulk_create(self, objs, batch_size=None):
>>> pre_bulk_create.send(sender=self.model, objs=objs, 
>>> batch_size=batch_size)
>>> res = super(MyModelQuerySet, self).bulk_create(objs, 
>>> batch_size)
>>> post_bulk_create.send(sender=self.model, objs=objs, 
>>> batch_size=batch_size)
>>> return res
>>>
>>>
>>> class MyModel(models.Model):
>>> #...
>>> objects = MyModelQuerySet.as_manager()
>>>
>>>
>>>
>>> This gives us a nice interface to handle all kind of changes regarding 
>>> `MyModel`
>>> Ou

Adding signals to bulk update/create operations

[Todor Velichkov]

Consider the following piece of code:

@receiver(pre_save, sender=MyModel)
def my_handler(sender, **kwargs):
instance = kwargs['instance']
if instance.verified:
do_something(instance)
else:
do_else(instance)



Its good, because it keeps `MyModel` decoupled from `do_something`

But there is one flaw. If we do:

MyModel.objects.filter(verified=False).update(verified=True)

we are screwed, `do_something` is not executed, and our models get 
out-of-sync.

If we try to get smart and manually fire the pre_save signal for each 
instance, we are gonna have a hard time.
Its gonna be slow.
And its gonna be memory inefficient.

We already experienced it in our app.

So our new approach is like this:

pre_bulk_update = Signal(providing_args=["queryset", "update_kwargs"])
post_bulk_update = Signal(providing_args=["update_kwargs",])

pre_bulk_create = Signal(providing_args=["objs", "batch_size"])
post_bulk_create = Signal(providing_args=["objs", "batch_size"])


class MyModelQuerySet(models.QuerySet):
def update(self, **kwargs):
pre_bulk_update.send(sender=self.model, queryset=self, 
update_kwargs=kwargs)
res = super(MyModelQuerySet, self).update(**kwargs)
# The queryset will be altered after the update call
# so no reason to send it.
post_bulk_update.send(sender=self.model, update_kwargs=kwargs)
return res

def bulk_create(self, objs, batch_size=None):
pre_bulk_create.send(sender=self.model, objs=objs, batch_size=
batch_size)
res = super(MyModelQuerySet, self).bulk_create(objs, batch_size)
post_bulk_create.send(sender=self.model, objs=objs, batch_size=
batch_size)
return res


class MyModel(models.Model):
#...
objects = MyModelQuerySet.as_manager()



This gives us a nice interface to handle all kind of changes regarding 
`MyModel`
Our example usage looks like this:


@receiver(pre_save, sender=MyModel)
def my_handler(sender, **kwargs):
instance = kwargs['instance']
if instance.verified:
do_something(instance)
else:
do_else(instance)


@receiver(pre_bulk_update, sender=MyModel)
def my_bulk_update_handler(sender, **kwargs):
update_kwargs = kwargs['update_kwargs']
if 'verified' not in update_kwargs:
# no change im interested in
# no need to take any action
return

queryset = kwargs['queryset']
pks_to_be_updated = queryset.values_list('pk', flat=True)
if update_kwargs['verified']:
do_something_bulk_update_implementation(pks_to_be_updated)
else:
bulk_update_do_else_implementation(pks_to_be_updated)


@receiver(pre_bulk_create, sender=MyModel)
def my_bulk_create_handler(sender, **kwargs):
objs = kwargs['objs']
group_1 = []
group_2 = []
for obj in objs:
if obj.verified:
group_1.append(obj)
else:
group_2.append(obj)

if group_1:
do_something_bulk_create_implementation(group_1)
if group_2:
bulk_create_do_else_implementation(group_2)



I think this turns out to be a very clean approach.
It help us use the most optimal strategy to handle the change.
So I'm sharing this with the community to check your feedback.
I believe if this gets into the Django Internals, it can be a very powerful 
tool.
It will lose power as a 3rd party app.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/94257dee-e88c-41ae-ab39-c8535d8e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.