Re: Model-level validation

2022-10-12 Thread Aaron Smith
I think the core developers who are making assertions about what is "accessible" and "makes sense" to newcomers would be well served by taking into account the actual experiences of newcomers. The expectation does not appear to align with reality. On Wednesday, October 12, 2022 at 1:19:56 AM

Re: Model-level validation

2022-10-12 Thread Jure Erznožnik
I'd like to chime in with this: There was a point in time when we ran into this issue and solved it with our own Model descendant. IMHO, I'm completely with Aaron on this: all the guts are there, just not being used. It took me quite a while to wrap my brain around the idea that validation

Re: Model-level validation

2022-10-10 Thread Shai Berger
I see two separate concerns here: 1) Should Django present to users the option to do validate-on-save by default? That is, should that option be visible -- in the form of a documented setting or an optional argument to save()? I tend to accept James' (and others) views and reasoning against

Re: Model-level validation

2022-10-09 Thread James Bennett
On Sat, Oct 8, 2022 at 8:32 PM Aaron Smith wrote: > Surely we can agree that *something* should happen here? The status quo >> is confusing, a footgun and a gotcha. If it's not Model's concern, then get >> it out of Model. >> > I've already said that I wish model-level validation hadn't been

Re: Model-level validation

2022-10-08 Thread Aaron Smith
> > And so I am still very strongly against trying to push a > model-layer-validation approach in Django, even optionally. > It already exists, though. `full_clean()` is a method on Model. CharFields on the model *already* have a notion of allowed choices. Validators are *already* an option

Re: Model-level validation

2022-10-08 Thread James Bennett
On Sat, Oct 8, 2022 at 8:44 AM Aaron Smith wrote: > The reason I don't want to use serializers or forms in celery tasks is > because validation should happen every time a model attribute is changed. > This pattern would mean more imports and boilerplate scattered around my > codebase any time I

Re: Model-level validation

2022-10-08 Thread Aaron Smith
James - these developers come from all over. They represent the random sample of people who end of working on or inheriting legacy django projects. Probably more Rails than anything else, but a few who probably worked with Hibernate or Node frameworks. For better or worse, it's who's using

Re: Model-level validation

2022-10-08 Thread James Bennett
On Fri, Oct 7, 2022 at 6:21 PM Aaron Smith wrote: > Mariusz - fair enough, I will consider my point made and apologies if it > came off too strong. FWIW it's not just my opinion, it's shared by every > developer (dozens) I've had this conversation with up until now. It's a > stark contrast that

Re: Model-level validation

2022-10-07 Thread Aaron Smith
Mariusz - fair enough, I will consider my point made and apologies if it came off too strong. FWIW it's not just my opinion, it's shared by every developer (dozens) I've had this conversation with up until now. It's a stark contrast that makes me wonder how aware the core developers / old

Re: Model-level validation

2022-10-07 Thread Mariusz Felisiak
> I am aware there are patterns to work around this in Django. My position is that skipping validation should be the rare edge case and not the easy naive path. Unless Django's stated purpose is to be a cute toy for making blogs, and robust infrastructure is off-label, but that's not what I see

Re: Model-level validation

2022-10-07 Thread Aaron Smith
Yes, every time you you get data from an untrusted source you must validate it. As well as *every time you change model attributes, ever*. There seems to be a widespread frame of mind in Django that validation is something you only need to do with data from a untrusted sources. As someone who

Re: Model-level validation

2022-10-07 Thread 'Barry Johnson' via Django developers (Contributions to Django itself)
I agree with James in several ways. Our large Django application does rather extensive validation of data -- but I would argue strongly against embedding that validation in the base instance.save() logic. (I would not argue against Django including a "ValidatingModel", derived from Model,

Re: Model-level validation

2022-10-07 Thread אורי
אורי u...@speedy.net On Fri, Oct 7, 2022 at 10:01 AM Carlton Gibson wrote: > > ... the duplication I was referring to is having both Forms and > Serializers do validation. > > That's a separate issue. > > Can we merge various aspects of DRF into Django, so that it better handles > building

Re: Model-level validation

2022-10-07 Thread Carlton Gibson
> ... the duplication I was referring to is having both Forms and Serializers do validation. That's a separate issue. Can we merge various aspects of DRF into Django, so that it better handles building JSON APIs? Yes, clearly. One step of that is better content type handling, another is

Re: Model-level validation

2022-10-06 Thread Aaron Smith
James - to clarify, the duplication I was referring to is having both Forms and Serializers do validation. I often work with web apps where data for the same model can arrive via user input, serializer, or created in some backend process e.g. Celery. If forms/serializers are your validation

Re: Model-level validation

2022-10-06 Thread James Bennett
On Thu, Oct 6, 2022 at 9:00 AM Aaron Smith wrote: > James - The problem with moving validation up the stack, i.e. to logical > branches from Model (Form, Serializer) is that you must duplicate > validation logic if your data comes from multiple sources or domains (web > forms *and* API endpoints

Re: Model-level validation

2022-10-06 Thread Aaron Smith
James - The problem with moving validation up the stack, i.e. to logical branches from Model (Form, Serializer) is that you must duplicate validation logic if your data comes from multiple sources or domains (web forms *and* API endpoints *and* CSVs polled from S3. Duplication leads to

Re: Model-level validation

2022-10-06 Thread Aaron Smith
Uri - that's a great upgrade path (or should I say, non-upgrade path). Agree with `VALIDATE_MODELS_BY_DEFAULT`. Rails also skips validations for some operations, like `update_column`, but they are prominently marked to use with caution, and the other ORMs i've used follow a similar pattern.

Re: Model-level validation

2022-10-06 Thread James Bennett
I see a lot of people mentioning that other ORMs do validation, but not picking up on a key difference: Many ORMs are designed as standalone packages. For example, in Python SQLAlchemy is a standalone DB/ORM package, and other languages have similar popular ORMs. But Django's ORM isn't

Re: Model-level validation

2022-10-05 Thread אורי
אורי u...@speedy.net On Thu, Oct 6, 2022 at 6:11 AM Aaron Smith wrote: > It sounds like there is little support for this being the default. But I'd > like to propose something that might satisfy the different concerns: > > 1) A `validate` kwarg for `save()`, defaulted to `False`. This

Re: Model-level validation

2022-10-05 Thread Curtis Maloney
FWIW +1 from me! -- C On Thu, 6 Oct 2022, at 14:11, Aaron Smith wrote: > It sounds like there is little support for this being the default. But I'd > like to propose something that might satisfy the different concerns: > > 1) A `validate` kwarg for `save()`, defaulted to `False`. This

Re: Model-level validation

2022-10-05 Thread Aaron Smith
It sounds like there is little support for this being the default. But I'd like to propose something that might satisfy the different concerns: 1) A `validate` kwarg for `save()`, defaulted to `False`. This maintains backwards compatibility and also moves the validation behavior users coming

Re: Model-level validation

2022-10-01 Thread Aaron Smith
David - All of your points are accurate. A usable ORM will probably never be perfectly safe, and none of the Django workarounds are particularly difficult. But requiring extra steps to get the save level of data safety as other ORMs will, just by nature of human nature and scale, make Django a

Re: Model-level validation

2022-10-01 Thread Aaron Smith
I'm glad you brought up tests, because automated tests for these issues are quite difficult and require tribal knowledge. A first-hand example from a while back: The application had over 90% test coverage, including all of the validation rules in the DRF serializer. One of those rules was that

Re: Model-level validation

2022-10-01 Thread David Sanders
I'm not really interested in debating whether the ORM validates or not but I thought it might be worth pointing out a few things that haven't been touched on yet: > It's not right. Design decisions are often neither outright right nor wrong but more tradeoffs of varying values. > The data store

Re: Model-level validation

2022-10-01 Thread Tim Graham
> Among the dozen of Django applications I have worked on, at 3 companies, not a single one was actually running any kind of validation. It has always been a mistake, 100% of the time, *never* the desired behavior. Besides not taking time to understand how Django works, it seems they weren't

Re: Model-level validation

2022-09-30 Thread Aaron Smith
Jorg, I do not believe it violates any separation of concerns. `full_clean()` is already a method on the Model class itself. The Model is already where all validation logic lives, except for the actual *triggering* of the validation. What I believe violates separation of concerns is that

Re: Model-level validation

2022-09-30 Thread Jörg Breitbart
@Aaron Oh well, if anecdotal personal evidence counts for you - here is mine: Working here since 2008 with Django in tons of projects in different positions. The project sizes were from small websites to big API-driven SPA cluster installations (with and w'o DRF). Ofc it is not all rainbows

Re: Model-level validation

2022-09-30 Thread Aaron Smith
Jorg, My observations come from the real world and are not hypothetical guesses. Among the dozen of Django applications I have worked on, at 3 companies, not a single one was actually running any kind of validation. It has always been a mistake, 100% of the time, *never* the desired behavior.

RE: Model-level validation

2022-09-30 Thread Danilov Maxim
-Original Message- From: django-developers@googlegroups.com [mailto:django-developers@googlegroups.com] On Behalf Of Jorg Breitbart Sent: Friday, September 30, 2022 10:12 AM To: django-developers@googlegroups.com Subject: Re: Model-level validation Hi there, I dont quite understand where

Re: Model-level validation

2022-09-30 Thread Jörg Breitbart
Hi there, I dont quite understand where the sudden fuzz about this comes from. I want to point out a few things, before going down the rabbit hole of competing high level interfaces, that ValidatedModel/UnvalidatedModel would introduce: - Django offers all validation needs as building

Re: Model-level validation

2022-09-29 Thread Adrian Torres
Hi, Regardless of what you consider ModelForms to be, the fact that validation doesn't happen at the model level is very jarring if you've ever used any other MVC framework, it was and still is one of the major pet peeves of Django for me, to the point where we do something similar to what Uri

Re: Model-level validation

2022-09-29 Thread Aaron Smith
I would also like everyone to know, my objective in starting this thread is to get the go-ahead to open a PR for this. I would like to contribute back. On Thursday, September 29, 2022 at 6:30:32 PM UTC-7 Aaron Smith wrote: > How about a new class, `ValidatedModel`, that subclasses `Model` and

Re: Model-level validation

2022-09-29 Thread Aaron Smith
Carl, All ORMs I have worked with allow you to bypass validations when necessary. Sometimes you have to. But the path of greatest naivety should be as safe as possible. I cannot even imagine how many lost hours and economic damages have occurred because the easy path is the dangerous path. I

Re: Model-level validation

2022-09-29 Thread Aaron Smith
How about a new class, `ValidatedModel`, that subclasses `Model` and does nothing more than call `full_clean()` on `save()`? This would be completely backwards compatible, would clearly communicate what it does, and when documented right next to `Model` make it fairly obvious that Model is

Re: Model-level validation

2022-09-29 Thread Carl Meyer
On Thu, Sep 29, 2022 at 6:19 PM Curtis Maloney wrote: > On Thu, 29 Sep 2022, at 14:29, Aaron Smith wrote: > > Why doesn't Django validate Models on save()? > > > The short answer is: backwards compatibility. > > Model level validation hasn't always been a thing, with Django initially > depending

Re: Model-level validation

2022-09-29 Thread Curtis Maloney
On Thu, 29 Sep 2022, at 14:29, Aaron Smith wrote: > Why doesn't Django validate Models on save()? The short answer is: backwards compatibility. Model level validation hasn't always been a thing, with Django initially depending primarily on Form validation. Since it hadn't _always_ been there,

Re: Model-level validation

2022-09-29 Thread Aaron Smith
Yes, I did search, and I did not find an answer to my question. If one is always supposed to use a ModelForm, why isn't that ModelForm functionality part of the Model? On Thursday, September 29, 2022 at 1:04:17 AM UTC-7 carlton...@gmail.com wrote: > Hi. > > I have to ask, did you search the

Re: Model-level validation

2022-09-29 Thread Aaron Smith
All I was able to find was that it was for "performance reasons", and I refuse to believe that a mature web framework like Django would prioritize performance (let's face it - even my millions-of-requests-per-day API service doesn't care about a few extra milliseconds here and there) over the

Re: Model-level validation

2022-09-29 Thread אורי
אורי u...@speedy.net On Thu, Sep 29, 2022 at 11:04 AM Carlton Gibson wrote: > > On Thursday, 29 September 2022 at 06:29:30 UTC+2 aa...@aaronsmith.co > wrote: > >> Why doesn't Django validate Models on save()? >> >> I am aware that full_clean() is called when using ModelForms. But most >> web

Re: Model-level validation

2022-09-29 Thread Carlton Gibson
Hi. I have to ask, did you search the history at all here? This has been discussed *several times* over the years. > Bringing View-level concepts such as forms down into celery tasks and management commands breaks separation of concerns... I think it's an error to think of forms (please