django api

[mahdi abolghasemi]

I have just started learning Django and am a beginner. I have a project 
related to cryptocurrency, and it is supposed that the white paper we 
import through the API will get the price and volume from the desired site 
(example: coinmarketcap.com), but no matter how much I search, I do not 
know how to implement this piece. Can you help me?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/38af0157-e7bf-4ead-b8b1-27697c38e00bn%40googlegroups.com.

Re: Ideas for a new DEP on long-term Django API compatibility

[Pascal Chambon]

Hello,

Just for the record, we spent a good time discussion this change (to bring
> inheritance of Admin Actions in line with Python’s expected inheritance
> rules). We reviewed the entire history of the feature. We explored an
> alternative approach, which would have maintained BC. We put the discussion
> to the mailing list, where further people reviewed it, and we came to a
> decision, as a group, without any objections being raised.


That's good and reassuring news; if so, that's precisely another occurrence
of "those reasons really need to be very good, and spelled out and
explained". Most Django users will never see farther than the release
notes, so if abrupt in-places changes occur without a rationale behind, it
*appears* reckless (whetever really happened in the background). If release
notes are considered too long already, wouldn't html tooltips/collapsibles
be useful, so that one may know more about the rational behind changes?

Tech debt is a very real thing.  Keeping it under control is vital to the
> pace of improvement and updates for any project, large, middling or small.
> Any project requires maintanence at minimum, just like owning a car or
> house.  If you buy a car and run it every day for five years, only filling
> up the gas tank and adding a splash of oil now and then,  you don't have
> any excuse to be upset at the mechanic or manufacturer for the size of the
> repair bill when things start breaking.  At least with software, its alot
> easier to get fixes out than with hardware.  You make a persuasive argument
> and have great passion, but the overall goal of enabling out of date and
> unsupported releases to remain around longer than they should be makes me
> feel really uncomfortable.  A spftware project is not buy once and forget,
> neither is a car.  And your package seems to encourage that lax practice to
> a level that I would be highly resistant at working at a company if I
> couldn't persuade management to prioritize keeping up to date and tech debt
> at a manageable level.
>

We agree on the necessity of updating code, and indeed DCP was never meant
to encourage slacking but to solve external dependency conflicts. The
latter - and the additional costs involved in solving them - are precisely
used by management to refrain from upgrading codebases.

As we're getting to the end of useful arguments here, I'd like to ask you
> to step back for a minute and take a calm look at what you're writing.
> Imagine you were receiving such messages rather than sending them. Would
> you want to spend more time collaborating with the person addressing you
> that way and to receive more such messages? Perhaps this is another reason
> why you aren't getting as much support as you'd hope.
>

My initial article, as well as most of this ML discussion, is not small
feature/bugfix request, but a rant on the whole deprecation process. The
aim is to raise awarenesss, to shake minds, to push points as deep as
possible, and to not let any falsehood or diversion slip by. As thus, it
uses all kinds of strong words, of emphases, of rhetorical figures, of
denounciations, to deliver the argumentation. I guess a rant will always
look aggressive, even if it only states the most scientific facts.

So I'm sorry if it sounded insulting, but personally I don't mind facing
adamant insistence, blunt statements, criticisms of any current state of
affairs (whatever the amount of effort one might have put into getting
there), and heated discussions, as long as they don't turn into ad hominem
attacks; in the excepts that you pointed out, I was mainly stating my frank
opinion on the (visibly incomplete) evidence that I was shown, and I would
only have sounded hypocritical by masking the seriousness these issues have
for me. Maybe some feel that Django as already having admirable commitment
regarding API stability, alas some other languages and frameworks have set
the bar much higher (farther than semantic versioning), and *they* are to
be used as references (although too extended compatibility is considered
harmful by some here).

Surely some of my words could have been better chosen ("punishing" shall be
understood as "putting at a disadvantage", and "little version" as "minor
version"), my bad, like some of yours ("throwing your toys" sounds
condescending, though I couldn't care less); but I feel that politeness
euphemisms and tone policing destroy the productiveness of crucial
controversies like this one, instead of helping them. I'd rather be called
names, but not see already-debunked untruths reappear in the conversation
("current behavior [...] *can't be fixed* without a hard compatibility
break", "*always* take the time to create deprecation paths"...); this
intellectual scrambling is what makes *me* stomach-sick, much more than old
unmaintained packages.

For example, Luke was open to amending the deprecation policy to better
> reflect reality. You should have suggested some concrete changes, erring on
> the

Re: Ideas for a new DEP on long-term Django API compatibility

[Dave Burkholder]

> I love where Django is headed. I love all of those breaking changes that 
have to happen so we're not perpetually stuck with decisions from 2005.

+100 to this!!

Django user since 1.6 running on python2.6. Upgraded to 1.7 and python3.4 
at the same time, and _that_ migration was hard. Everything since then has 
been a cakewalk. 

Mega-props to the caretakers of Django. Carry on...!



On Monday, August 19, 2019 at 11:51:33 AM UTC-4, Patryk Zawadzki wrote:
>
> As someone who has spent over 12 years working with Django, I'd like to 
> offer an opposite point of view:
>
> I love where Django is headed. I love all of those breaking changes that 
> have to happen so we're not perpetually stuck with decisions from 2005.
>
> What I truly miss is strong static typing support that would tell that 
> when my code is not going to work before I even begin to test it. With 
> static types and tools like mypy and pyright upgrading dependencies is 
> usually a matter of a single afternoon.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6b6f6b14-c3cc-4ba9-9d77-92ecaab23e11%40googlegroups.com.

Re: Ideas for a new DEP on long-term Django API compatibility

[Patryk Zawadzki]

As someone who has spent over 12 years working with Django, I'd like to 
offer an opposite point of view:

I love where Django is headed. I love all of those breaking changes that 
have to happen so we're not perpetually stuck with decisions from 2005.

What I truly miss is strong static typing support that would tell that when 
my code is not going to work before I even begin to test it. With static 
types and tools like mypy and pyright upgrading dependencies is usually a 
matter of a single afternoon.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f9eaf95a-8611-4715-a4bf-6aed7a284b3f%40googlegroups.com.

Re: Ideas for a new DEP on long-term Django API compatibility

[Aymeric Augustin]

Hello Pascal,

As we're getting to the end of useful arguments here, I'd like to ask you to 
step back for a minute and take a calm look at what you're writing. Imagine you 
were receiving such messages rather than sending them. Would you want to spend 
more time collaborating with the person addressing you that way and to receive 
more such messages? Perhaps this is another reason why you aren't getting as 
much support as you'd hope.

You did your best to provide nuanced feedback on a debatable 
backwards-compatibility issue — current behavior is obviously wrong but can't 
be fixed without a hard compatibility break:
> some wild guess / wishful thinking

You spent thousands of hours thinking about what would be best for users of the 
framework — getting occasional thanks and only occasional abuse, as we're lucky 
to have a positive community:
> what is practical for core developers is NOT the same as what is practical 
> for users

You don't need backwards compatibility because you're comfortable living on the 
edge but you always take the time to create deprecation paths according to the 
compatibility policy:
> even if it means creating a task force just for keeping a slight eye on 
> compatibility shims

You do your best to discuss positively with an agressive contributor who 
started with a strongly-worded blog post and doesn't seem to process the 
responses he's getting:
> All that to face red herrings


Here's literal belittling:
> *your* little version of Django


And the cherry of the cake, stating core devs are secret sadists:
> Punishing users


This isn't how we hold discussions here. You need to take a clue from how other 
people interact on this mailing list. Then you can put your energy and ideas to 
good use for improving Django.

>From a pragmatic perspective, putting people off isn't a good way to get them 
>to cooperate with you.

For example, Luke was open to amending the deprecation policy to better reflect 
reality. You should have suggested some concrete changes, erring on the side of 
caution. Instead you chose to repeat the same argument, except more 
agressively. This doesn't get us any closer to a documentation patch. In fact 
this reduces the likelihood that someone will choose to spend time writing that 
patch.

You last email sounds like you're throwing your toys and walking away because 
you didn't get full agreement with your position. However, that outcome was 
largely predictable when you were bringing up a position that diverges so much 
from the current practice. To create progress, it's up to you to create 
consensus around small steps and to create the corresponding pull requests, in 
that order.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/E1632450-C0E8-410F-A3BB-03B5FAD94C13%40polytechnique.org.

Re: Ideas for a new DEP on long-term Django API compatibility

[Jason Johns]

Pascal, I don't think anyone here disagrees with your overall goal to 
reduce the breaking changes.  As someone that got handed a 1.2 project with 
zero tests and updated it to 1.8, good test coverage to lay the foundation 
for reviving an ambitious project that went comatose before being 
prioritized again, the amount of breakage and updating required was 
surprisingly small.  It took about a month or so to do, and a quarter of 
that was setting up staging and prod environments on AWS.  The core devs 
here have done a great job, in my opinion, of keeping the breaking changes 
to a minimum, and making them in a thoughtful manner with an eye to the 
future.

Tech debt is a very real thing.  Keeping it under control is vital to the 
pace of improvement and updates for any project, large, middling or small.  
Any project requires maintanence at minimum, just like owning a car or 
house.  If you buy a car and run it every day for five years, only filling 
up the gas tank and adding a splash of oil now and then,  you don't have 
any excuse to be upset at the mechanic or manufacturer for the size of the 
repair bill when things start breaking.  At least with software, its alot 
easier to get fixes out than with hardware.  You make a persuasive argument 
and have great passion, but the overall goal of enabling out of date and 
unsupported releases to remain around longer than they should be makes me 
feel really uncomfortable.  A spftware project is not buy once and forget, 
neither is a car.  And your package seems to encourage that lax practice to 
a level that I would be highly resistant at working at a company if I 
couldn't persuade management to prioritize keeping up to date and tech debt 
at a manageable level.

Open source has a pretty low barrier to entry, and django projects like the 
ones you listed on django packages are reflective of that.  Its easy to 
make a package and put it out, but maintaining and iterating and releasing 
take alot more in terms or time, effort and emotional state.  Fortunately, 
Python as a whole seems to be much easier on package devs than the JS 
community, but as I'm sure Carlton, Adam, Tim, and anyone that's published 
a package with requests for support can attest, it can get exhausting.  
That's when packages just die on the vine.  Some are lucky to 
be resuscitated by another dev taking over, or forking the original, but 
that doesn't happen for the majority.  Deliberately going out of your way 
to ensure that old code which hasn't been looked at in however long and was 
last working on two LTS releases ago gives me a very squicky feeling in my 
stomach.  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/097ecac4-0596-40d4-a069-7e8d2c879ce9%40googlegroups.com.

Re: Ideas for a new DEP on long-term Django API compatibility

[Carlton Gibson]

Hi Pascal. 

> On 17 Aug 2019, at 18:21, Pkl  wrote:
> 
> In just 5 lines of discussion...

Just for the record, we spent a good time discussion this change (to bring 
inheritance of Admin Actions in line with Python’s expected inheritance rules). 
We reviewed the entire history of the feature. We explored an alternative 
approach, which would have maintained BC. We put the discussion to the mailing 
list, where further people reviewed it, and we came to a decision, as a group, 
without any objections being raised. 

We didn’t make this decision lightly. 

On balance, we felt the breaking change was justified by the ease of adaption, 
the fact nobody could identify a single use-case of the existing behaviour in 
the wild — and to be fair, there have not been issues raised because of it, 
which we might have expected — and finally, that Django would be better for the 
change.  

I’m sorry that we have a different approach than the one you’ve asked for, but 
hopefully you can accept the reasons that people have given, and, more 
importantly, that decisions are made in good faith, for the best outcome that 
we, as a group, can find.

Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1E067ACA-977F-42F3-A875-B294EC7EF0BF%40gmail.com.

Re: Ideas for a new DEP on long-term Django API compatibility

[Adam Johnson]

east two feature releases*" is itself
>> still misleading, since at least one recent major breakage violates it
>> <https://docs.djangoproject.com/en/2.2/releases/2.2/#admin-actions-are-no-longer-collected-from-base-modeladmin-classes>
>> ; *A pronouncement of the core dev team would be needed here**: is this
>> behaviour change a procedure error? Or is it really the new policy, to
>> allow changes which create immediate silent errors, and prevent third-party
>> compatibility shims from getting implemented?*
>>
>> Having "only one obvious way to day it" is a useful and common commitment
>> in Python, but please note the "obvious". You won't have "one way to do
>> it", ever. If people want to use sys.stdout.write(), or do a libc-cffi call
>> to output text to console, instead of just calling print(), they will do
>> so. And compatibility shims don't change anything on this matter: the
>> "obvious", "standard" way of doing is just the one mentioned in official
>> docs and which doesn't trigger DeprecationWarnings.
>>
>>
>>
>>
>>> *Second,* there were things I wanted to say that might be useful to you
>>> (Pascal) in understanding why you are not likely to get very far with your
>>> current approach.
>>>
>>> Your exaggerated tone is not likely to help you. You talk about Django's
>>> compatibility breakages as "*ruining the day of thousands of anonymous
>>> plugin/website maintainers all over the world*". When we make these
>>> kind of changes, we always do it with very clear and detailed upgrade
>>> notes, and almost always with several releases emitting deprecation
>>> warnings, as per our policy. And no-one is forced to upgrade immediately.
>>> Those who value stability above other things can use LTS, which gives them
>>> a *3 year release cadence,* not 8 months, which is far from excessively
>>> fast compared to other projects.
>>>
>>
>>
>> Imho dependency hell is absolutely unrelated to detailed release notes or
>> LTS versions. Dependency hell is just the fact that, when each minor
>> version breaks compatibility, the app ecosystem inevitably explodes,
>> especially in its outskirts (i.e the least popular packages), into a myriad
>> fragments, each targeting specific django versions.
>> Your project remains on 1.11 LTS ? Nice, but one of your dependency (and
>> its latest security bugfixes) require a new feature of Django 2.0; while
>> another dependency is stuck on Django 1.10, either because the maintainer
>> vanished, or because he is himself trapped by the requirements of a
>> subdependency.
>>
>> That's why Semantic Versioning was invented: so that developers can
>> gradually update their own codebase (since each minor version brings
>> improvements and deprecation warnings), while still ensuring all
>> dependencies keep working under the same major versions (the project just
>> has to install the latest minor version). The concept of "lax semantic
>> versioning" sounds like a joke, it destroys the very purpose of this
>> versioning scheme; would you put a "laxly waterproof watch" under water?
>>
>> You mention Mezzanine, and indeed this CMS, being quite monolithic,
>> should not have big troubles with the Django compatibility policy. More
>> generally, as long as one sticks to a bunch of big packages (Mezzanine,
>> Django-rest-framework, Django-extensions...), things work fine. Dependency
>> hell occurs when you use modular architectures, and want to cherry pick the
>> most relevant modules for your needs, even those who haven not invested
>> into custom shims and multi-version testing, or which have not received
>> updates for a few years.
>> Such modular architectures are precious, they ensure that users don't get
>> trapped by feature choices of underlying frameworks; but they need this
>> extra bit of carefulness about softare compatibility.
>>
>> Take for example Django-CMS, a dozen external content type plugins
>> (ckeditor, restructuredtext, videos, musics, image galleries...), a blog
>> engine and its bridge (Zinnia), some utilities (easy-thumbnails,
>> django-filer...); and there you are, struggling to make it all work despite
>> large-impact breakages of the Django API. All this, not to do rocket
>> science, just to build what we would call a simple website.
>>
>> Let's note that Mezzanine had to fork its own dependencies, by the way,
>> and in big part for what reason?
>>
&

Re: Ideas for a new DEP on long-term Django API compatibility

[Tim Graham]

onymous 
>> plugin/website maintainers all over the world*". When we make these kind 
>> of changes, we always do it with very clear and detailed upgrade notes, and 
>> almost always with several releases emitting deprecation warnings, as per 
>> our policy. And no-one is forced to upgrade immediately. Those who value 
>> stability above other things can use LTS, which gives them a *3 year 
>> release cadence,* not 8 months, which is far from excessively fast 
>> compared to other projects.
>>
>
>
> Imho dependency hell is absolutely unrelated to detailed release notes or 
> LTS versions. Dependency hell is just the fact that, when each minor 
> version breaks compatibility, the app ecosystem inevitably explodes, 
> especially in its outskirts (i.e the least popular packages), into a myriad 
> fragments, each targeting specific django versions. 
> Your project remains on 1.11 LTS ? Nice, but one of your dependency (and 
> its latest security bugfixes) require a new feature of Django 2.0; while 
> another dependency is stuck on Django 1.10, either because the maintainer 
> vanished, or because he is himself trapped by the requirements of a 
> subdependency.
>
> That's why Semantic Versioning was invented: so that developers can 
> gradually update their own codebase (since each minor version brings 
> improvements and deprecation warnings), while still ensuring all 
> dependencies keep working under the same major versions (the project just 
> has to install the latest minor version). The concept of "lax semantic 
> versioning" sounds like a joke, it destroys the very purpose of this 
> versioning scheme; would you put a "laxly waterproof watch" under water? 
>
> You mention Mezzanine, and indeed this CMS, being quite monolithic, should 
> not have big troubles with the Django compatibility policy. More generally, 
> as long as one sticks to a bunch of big packages (Mezzanine, 
> Django-rest-framework, Django-extensions...), things work fine. Dependency 
> hell occurs when you use modular architectures, and want to cherry pick the 
> most relevant modules for your needs, even those who haven not invested 
> into custom shims and multi-version testing, or which have not received 
> updates for a few years. 
> Such modular architectures are precious, they ensure that users don't get 
> trapped by feature choices of underlying frameworks; but they need this 
> extra bit of carefulness about softare compatibility.
>
> Take for example Django-CMS, a dozen external content type plugins 
> (ckeditor, restructuredtext, videos, musics, image galleries...), a blog 
> engine and its bridge (Zinnia), some utilities (easy-thumbnails, 
> django-filer...); and there you are, struggling to make it all work despite 
> large-impact breakages of the Django API. All this, not to do rocket 
> science, just to build what we would call a simple website.
>
> Let's note that Mezzanine had to fork its own dependencies, by the way, 
> and in big part for what reason?
>
> *"At the time of grappelli_safe's creation, Grappelli was incorrectly 
> packaged on PyPI, and had also dropped compatibility with Django 1.1 -- 
> grappelli_safe was therefore created to address these specific issuesAt the 
> time of filebrowser_safe's creation, FileBrowser was incorrectly packaged 
> on PyPI, and had also dropped compatibility with Django 1.1 -- 
> filebrowser_safe was therefore created to address these specific issues."*
> Nice, more fragmentation.
>
> Now, I can't talk about percentages. I just *estimate *that thousands of 
> projects (a part of the total) are relying on modular frameworks like 
> django-cms, or must make lots of small django apps play together. That's 
> for these people that days are wasted, even if they don't realize how 
> unneeded all this work duplication is. For sure, dependency hell sounds 
> like a made-up or exaggerated concept, until you have to deal with it; to 
> take a harsh metaphor, people of Europe see hippos as peaceful giants, but 
> that's not the opinion of the 3000+ people they kill each year in Africa.
>
> If there is an *exageration *somewhere, I bet it's rather on all that 
> "*backwards 
> compatibility is too hard/costly/dangerous*" speech I'm being served. My 
> experiments at the contrary showed that backwards compatibility was easy to 
> setup and maintain, provided you have some tooling under the hand (tooling 
> that I did implement, like import aliasers). Why be worried about "*Django 
> codebase (becoming) a pile of backwards compatibility hacks and cruft*", 
> when compat-patchers precisely separate the compatibility concern from the 
>

Re: Ideas for a new DEP on long-term Django API compatibility

[Pkl]

y update their own codebase (since each minor version brings 
improvements and deprecation warnings), while still ensuring all 
dependencies keep working under the same major versions (the project just 
has to install the latest minor version). The concept of "lax semantic 
versioning" sounds like a joke, it destroys the very purpose of this 
versioning scheme; would you put a "laxly waterproof watch" under water? 

You mention Mezzanine, and indeed this CMS, being quite monolithic, should 
not have big troubles with the Django compatibility policy. More generally, 
as long as one sticks to a bunch of big packages (Mezzanine, 
Django-rest-framework, Django-extensions...), things work fine. Dependency 
hell occurs when you use modular architectures, and want to cherry pick the 
most relevant modules for your needs, even those who haven not invested 
into custom shims and multi-version testing, or which have not received 
updates for a few years. 
Such modular architectures are precious, they ensure that users don't get 
trapped by feature choices of underlying frameworks; but they need this 
extra bit of carefulness about softare compatibility.

Take for example Django-CMS, a dozen external content type plugins 
(ckeditor, restructuredtext, videos, musics, image galleries...), a blog 
engine and its bridge (Zinnia), some utilities (easy-thumbnails, 
django-filer...); and there you are, struggling to make it all work despite 
large-impact breakages of the Django API. All this, not to do rocket 
science, just to build what we would call a simple website.

Let's note that Mezzanine had to fork its own dependencies, by the way, and 
in big part for what reason?

*"At the time of grappelli_safe's creation, Grappelli was incorrectly 
packaged on PyPI, and had also dropped compatibility with Django 1.1 -- 
grappelli_safe was therefore created to address these specific issuesAt the 
time of filebrowser_safe's creation, FileBrowser was incorrectly packaged 
on PyPI, and had also dropped compatibility with Django 1.1 -- 
filebrowser_safe was therefore created to address these specific issues."*
Nice, more fragmentation.

Now, I can't talk about percentages. I just *estimate *that thousands of 
projects (a part of the total) are relying on modular frameworks like 
django-cms, or must make lots of small django apps play together. That's 
for these people that days are wasted, even if they don't realize how 
unneeded all this work duplication is. For sure, dependency hell sounds 
like a made-up or exaggerated concept, until you have to deal with it; to 
take a harsh metaphor, people of Europe see hippos as peaceful giants, but 
that's not the opinion of the 3000+ people they kill each year in Africa.

If there is an *exageration *somewhere, I bet it's rather on all that 
"*backwards 
compatibility is too hard/costly/dangerous*" speech I'm being served. My 
experiments at the contrary showed that backwards compatibility was easy to 
setup and maintain, provided you have some tooling under the hand (tooling 
that I did implement, like import aliasers). Why be worried about "*Django 
codebase (becoming) a pile of backwards compatibility hacks and cruft*", 
when compat-patchers precisely separate the compatibility concern from the 
rest? 
As for the fear of "non local reasoning" expressed in other posts, I'm 
sorry, this doesn't apply here; compatibility fixers don't do evil things, 
like renaming some objects and removing others. They only ADD shims, they 
are activated on demand, they output logs and warnings to keep users 
informed, and if a shim breaks, you have maybe 99% chances the faulty 
wrapper/utility will appear in the traceback. So there is no backstabbing 
here.

By the way, some points I should have added to my initial article:

   - The most widely breaking changes are, generally, the easiest to fix. 
   Moving a core function/module will break every app, and can be fixed in <10 
   lines of code (tests included). Changing the corner-case behaviour of an 
   ORM query method is hard to maintain, but very few users should be impacted 
   by this change. That's why deprecation periods should be set on a 
   case-by-case basis, and maybe 90% of breakages can be avoided just with 
   some dozens of tiny long-term shims.
   - If the framework code evolves too fast for its own maintainers to cope 
   with compatibility shims, then it probably also evolves too fast for plugin 
   and project maintainers to keep track, too. In this case, it should be 
   called alpha-stage software, and people should wait for it to stabilize.


Thats why I don't buy your "*Quite quickly you get to a point where the 
quantity of cruft, and the need to support all the different legacy ways, 
means that adding new things becomes very difficult, and development would 
have stagnated and quality dr

Re: Ideas for a new DEP on long-term Django API compatibility

[Luke Plant]

Hi Pascal,

I know this is a very late reply, but there have been some things going 
round my head that I thought I should get down.


*First,* some positive changes I think we can make from your suggestions:

1. I think in our release notes, for breaking changes that might need 
some justification, we should link to the tickets and/or django-devs 
discussions behind them. Adding full reasoning in the notes themselves 
would likely bloat them far too much, but a link could be helpful for 
anything which is not obviously necessary.


2. We should change the wording in our API stability docs 
, which 
does indeed initially sound like a promise never to break working code 
that follows docs:


   Django promises API stability and forwards-compatibility since
   version 1.0.

We do have this line later on:

   If, for some reason, an API declared stable must be removed or
   replaced, it will be declared deprecated but will remain in the API
   for at least two feature releases. Warnings will be issued when the
   deprecated method is called.

But we do make changes at a rate faster than that document would make 
you think.


I suggest that as a contrast to the existing text on that page, we also 
point out that we are continually improving Django, and have a 
"(eventually) one way to do it" policy regarding how we plan APIs, which 
means we will remove old things, while always providing better 
alternatives. I'm happy to write a patch for this.


*Second,* there were things I wanted to say that might be useful to you 
(Pascal) in understanding why you are not likely to get very far with 
your current approach.


Your exaggerated tone is not likely to help you. You talk about Django's 
compatibility breakages as "/ruining the day of thousands of anonymous 
plugin/website maintainers all over the world/". When we make these kind 
of changes, we always do it with very clear and detailed upgrade notes, 
and almost always with several releases emitting deprecation warnings, 
as per our policy. And no-one is forced to upgrade immediately. Those 
who value stability above other things can use LTS, which gives them a 
*3 year release cadence,* not 8 months, which is far from excessively 
fast compared to other projects.


My involvement in Django these days is primarily not as a core 
contributor, but as a user and library maintainer, and I've also 
contributed many patches to different 3rd party Django projects to fix 
them for Django upgrades. The impression that you give of Django being a 
continuously shifting sand is very far from my experience. The core 
features and structure of Django have remained remarkably stable over 
its history. A few import changes here and there are very easy to fix 
(and, more importantly, noisy and easy to diagnose), and other changes 
are usually straightforward. Normally upgrades to libraries I maintain 
to support the latest Django version require very little work, sometimes 
zero, and often I'm able to support 4 or 5 Django versions without a 
problem. Using something like django-compat-patcher, or some of the 
other similar libraries, would add more complexity than the few lines of 
code I need otherwise.


Even larger, more complex Django projects like Mezzanine, which use 
quite a few hacks and internals, are often able to span across two LTS 
versions.


With all this in mind, it's difficult to see how we are making so many 
people's lives miserable.


Instead of debating in abstract, or answering your points one by one, I 
thought a few illustrations might help you see the benefits of Django's 
approach. The first is the longest, it might be of historical interest 
to others.



   1. A 14 year old Django project

I started work on my first Django project in September 2005 
, 
and it went live in March 2006. It has been running continuously since 
then - perhaps by now it can claim to be one of the oldest continuously 
running Django sites? I'd be interested to know if anyone can beat it...


Here's a snippet from one of my models.py nearly 14 years ago:


   class Message(meta.Model):
    fromUser = meta.ForeignKey(User,
        verbose_name="from user",
        related_name="messageSent"
    )
    toUser = meta.ForeignKey(User,
        verbose_name="to user",
        related_name="messageReceived")
    time = meta.DateTimeField("At")
    text = meta.TextField("Message")
    box = meta.PositiveSmallIntegerField("Message box",
        choices = MESSAGE_BOXES)

    def __repr__(self):

        #return str(self.id)
        return "[" + str(self.id) + "] to " + repr(self.get_toUser())  + " from 
" + repr(self.get_fromUser())

    class META:

        admin = meta.Admin(
            list_display = ('toUser', 'fromUser', 'time')
        )
  

Re: Ideas for a new DEP on long-term Django API compatibility

[Pkl]

Hello everyone,


(apologies in advance for the long post, but there are lots of aspects to 
dig here, it's not for the pleasure of writing but a necessary to progress 
on the relevance of a DEP)

*"As you can see, there's more than "blinding self-absorption" and "harmful 
psychological bias" here. Your freecodecamp article makes valid points. It 
also misses completely what's hard in maintaining backwards compatibility. 
I wish you hadn't found it useful to insult the work put into managing 
backwards-compatibility over the years and the people who did it."*

I'd rather address that immediately: as you read in my article, "blinding 
self-absorption", "harmful psychological bias" and other strong words were 
all aimed at something very specific, the "feeling of return to purity" 
that a person could have when destroying compatibility shims. I persist and 
sign regarding this one: ruining the day of thousands of anonymous 
plugin/website maintainers all over the world should *always* be made under 
compulsion of heavy constraints, and with regrets; never lightheartedly. 

Now, maybe my attempt at explaining the roots of nowadays' mass breakages 
by a cultural problem are all wrong (my bad), and there are other reasons 
behind the *apparently* reckless compatibility destructions that I witness 
left and right. But what are they? Since it's not technical inability, 
since it's not laziness, since it's not ideology, what makes that (as far 
as I'm concerned) the Django ecosystem fails on that aspect, contrary to 
loads of others "big" frameworks, runtimes, libraries etc? 

If it's "we don't have enough resources", it's all understandable, but at 
least let's make it clear. The weirdest, in the whole story, is the global 
idea that everything is fine, that compatibility is already a "major 
concern" and treated so, that maybe it can be improved a little bit but no 
need to push it. It's not that way I see it. And a problem can't be 
addressed if it's not first recognized as so.

Now, I know that it requires a good amount of diplomacy to criticize a 
behaviour, or here a whole policy, without concerned people feeling 
attacked. I don't have this kind of talent (and probably won't ever), so 
all I can do is be honest with my view on the situation, and bring attempts 
at solutions. It's precisely because Django is a great piece of software 
(with its omnipresent lazy-objects, it's automatic admin and model forms, 
its migration system, its focus on security headers...) that having it 
scarifying itself is unbearable. So I apologize in advance for all the 
criticisms that expand below, but without them there'd be no need for any 
DEP or change in the first place.

I also want to make this clear: I didn't mean that breaking changes were 
never justified, just that they had to be the last option, and thoroughly 
explained. When the AngularJS team, after the years of hype, admitted that 
the whole designed was flawed, reset the whole framework as Angular version 
2, and made great efforts to ease part-by-part migration towards the new 
behaviour, I felt unlucky but not angry. When trivial changes ruined tons 
of pluggable Django apps that did one thing and did it well, it was a 
different matter.

Before answering remarks, I think it's worth summarizing what the "problem" 
is. This is not lightly that I used the words "dependency hell". My 
experience with Django, purely regarding compatibility, has been abysmal 
compared to lots of other ecosystems (in C/C++, PHP, Jquery...) that I have 
crossed, and which didn't have the malleability of Python; I see several 
cumulated reasons explaining that :

- The pace of breakages: an arm's long list of new breaking changes 
introduced every 8 months, dropping at the same time all shims older than 
2-3 years, is imo a too fast breakage pace, even by the Web's standard. 
When changes are mainly aesthetical, compatibility shims had better remain 
for very long periods of time, else the benefit/harm balance is just 
indefensible.
- More importantly, the *scope* of breakages: as long as it's only about 
fixing corner cases of the framework, and only a tiny portion of users is 
impacted (those who abused counter-intuitive and unpythonic behaviours), 
one can understand, and just hope not to be shot by the next upgrade; but 
here, we're dealing with changes that deliberately break about every 
existing application, even those which scrupulously followed official 
tutorials: ForeignKey's "on_delete" argument killed most migration files 
when becoming mandatory, moving url-related primitives left and right was 
as destructive as expected...
- The growing "less batteries included" philosophy. Valueable contrib 
modules were outsourced from the codebase, and other pluggable applications 
(likes CMS apps) follow this trend, putting all valuable features into 
external plugins. This "minimal core" philosophy would work fine if main 
applications counter-balanced it with a more delicate ap

Re: Ideas for a new DEP on long-term Django API compatibility

[Florian Apolloner]

Hello World,

On Sunday, June 30, 2019 at 9:24:05 AM UTC+2, Aymeric Augustin wrote:
>
> As an experiment, let's just consider the first backwards-incompatible 
> change from the latest release: 
> https://docs.djangoproject.com/en/2.2/releases/2.2/#admin-actions-are-no-longer-collected-from-base-modeladmin-classes
>  
> 
>

I think Aymeric is making a good point here on why any project like 
django-compat-patcher is ultimately doomed to be a failure. You simply 
cannot magically "fix" changes like this without knowing user intent. I can 
also remember the change from get_query_set to get_queryset (1.6 IIRC), you 
simply cannot magically patch that to work with multiple sublcasses and 
dynamic attributes. There will __always__ be situations where this will 
break in horrible and subtle ways. I'd argue that this would be worse then 
not being able to use a project that is not updated for a while. 

Pakal wrote in his initial mail:

> With *less* work than currently, except small changes in procedures, we 
> can revive the majority of existing packages (except python2vs3 troubles), 
> ...
>

This is a bold argument for which I'd like to see some proof, also *less* 
work for whom? 

And last but not least, if one assumes all your arguments hold true, then 
why isn't django-compat-patcher used by existing 3rd party libraries (At 
least according to public usage 
https://github.com/pakal/django-compat-patcher/network/dependents)? Either 
the usecase you are suggesting isn't as strong as you make it to be or 3rd 
party packages are maintained well enough for this to be not a problem? 
Personally I also think that a package that wasn't updated since 2015 
should probably not be magically patched to theoretically work on a current 
Django.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7b076492-0ad7-4a06-8bef-008cd6fd91f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ideas for a new DEP on long-term Django API compatibility

[Aymeric Augustin]

Hello Pakal,

If I understand correctly, you're proposing that:

1. The Django project should maintain a higher level of backwards compatibility.
2. This would be easier to achieve outside the main codebase, in a submodule or 
in a separate repository.
3. This would reduce the amount of work required for maintaining Django 
(triaging tickets, fixing bugs, adding features).

Backwards-compatibility is a major concern for the development of Django, as 
you can see from in contribution documentation, in the archives of this mailing 
list, in DEPs, etc. Maintaining more compatibility with the same effort or less 
should be uncontroversial. Now the question is to get consensus that your 
proposal will indeed achieve the stated goals.

I'm quite skeptical of the "same effort or less". The more behaviors Django 
maintains — and you're proposing essentially to maintain all behaviors that 
existed at all previous releases — the more complexity accrues. You can't solve 
that just by shuffling deprecation warnings or compatibility imports in another 
module (and it seems to me that it would be harder to maintain, for the reason 
the Python ecosystem usually eschews monkey-patching: non local effects). 
Keeping old ways around forever increases the barrier to mastering Django for 
people who haven't been writing Django code for ten years like you and me. It 
also increases the scope on which compatibility must be maintained, and that 
can get in the way of making Django better.

One way to frame the debate is: does Django aim at being a fossilizing 
ecosystem or a living one? Up to this point, we've chosen to maintain a living 
ecosystem. At a given point in time, all maintained libraries support 2 or 3 
versions of Django (latest, previous, LTS). Things change a bit; unmaintained 
libraries that don't adapt fall out of favor; new libraries build upon the 
experience of previous ones and try to do better. One can see this as progress 
and a way to keep up with the moving web ecosystem. One can also see this as 
useless churn. As you call it "walk or die", it's pretty clear which camp 
you're in :-) I'd say there's truth in both!

As an experiment, let's just consider the first backwards-incompatible change 
from the latest release: 
https://docs.djangoproject.com/en/2.2/releases/2.2/#admin-actions-are-no-longer-collected-from-base-modeladmin-classes
 

 (I'm skipping the database backends changes because they're explicitly out of 
the backwards-compatibility policy, but we started documenting them for the 
maintainers of third-party database backends.) This is a good example. Django 
behaved in a non-Pythonic way; it didn't respect the Principle of Least 
Astonishment. Since we believe Django is alive and will have infinitely more 
future users than past users, we make the change. How can we maintain 
backwards-compatibility for this? We can try to detect if subclasses have all 
the actions of their parent classes and, if not, raise a warning. But then we 
raise inappropriate deprecations warnings in legitimate use cases where a 
subclass mustn't have some actions of its parent class, which is probably the 
use case for which this change was requested.

Experimenting with a third party module like you did is absolutely the way to 
go. You're already ahead of the usual advice :-) Keep in mind that the barrier 
for making a third-party project an official Django project or to merging it in 
Django is very high. With 1 fork and 2 stars, django-compat-patcher doesn't 
pass it yet.

Writing a DEP to formalize arguments on both sides is a valid idea. Don't 
forget the other side. In your freecodecamp article, you're spending two lines 
on the downsides of Compat Patchers, and this is not enough. How would Django 
communicate to users which backwards-incompatible changes are covered by Compat 
Patchers and which aren't? How would we respond to users asking for a Compat 
Patcher for a backwards-incompatible change for which it's impossible? 
Eventually, will every library document the list of Compat Patchers it requires 
or it's incompatible with?

As you can see, there's more than "blinding self-absorption" and "harmful 
psychological bias" here. Your freecodecamp article makes valid points. It also 
misses completely what's hard in maintaining backwards compatibility. I wish 
you hadn't found it useful to insult the work put into managing 
backwards-compatibility over the years and the people who did it.

Best regards,

-- 
Aymeric.



> On 29 Jun 2019, at 13:09, Pkl  wrote:
> 
> Hello everyone,
> 
> I'm planning on writing a PEP on long-term API stability for Django.
> Most of the rationale behind this kind of commitment is described here : 
> https://www.freecodecamp.org/news/api-stability-is-cheap-and-easy/
> 
> The idea is that the django ecosystem has suffered a lot, over the past 
> decade, from the 

Ideas for a new DEP on long-term Django API compatibility

[Pkl]

Hello everyone,

I'm planning on writing a PEP on long-term API stability for Django.
Most of the rationale behind this kind of commitment is described here : 
https://www.freecodecamp.org/news/api-stability-is-cheap-and-easy/

The idea is that the django ecosystem has suffered a lot, over the past 
decade, from the numerous breakages happening at each (minor) release. I 
could write pages just about what "outsiders" like me endured regarding 
neverending URL patterns changes.

Lots of pluggable apps, and their own plugins and bridges, have become 
incompatible with each other due to sticking to different django versions, 
or to lack of Tox-like cross-version testing. Some undocumented breaking 
changes (like classes replaced by other ones but missing functionality, or 
laziness introduced without autoloading measures for backwards 
compatibility (RequestContext?)) also hindered misc projects, as seen in 
their bugtrackers.

With *less* work than currently, except small changes in procedures, we can 
revive the majority of existing packages (except python2vs3 troubles), 
currently maintained or not, and make upgrades much smoother, which will 
also improve the overall security of django-backed sites (since users won't 
fear upgrades anymore). We can thus reach the same regard for compatibility 
as most industrial-size ecosystems, without sacrificing the cleanness of 
the codebase, nor its maintainability.

For this, imho the best way is just have to include in Django official 
support for a companion library like 
https://github.com/pakal/django-compat-patcher , with which I've had 
immense success

There are several ways of doing this, and for this more thoughts are 
necessary:

- putting compatibility sets in the repository, as a django.compt submodule 
for example?
- putting them in an external repository, but strongly advising them in 
tutorials and otehr docs, like a prime "official project" of django?

There are probably aspects of the issue that I'm missing, as I'm not into 
django "core" development.
So your feedbacks are more than welcome!

Thanks,
regards,
Pakal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1d2f1399-3310-4b36-ae74-9581a0695002%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.