Re: Vendoring multipledispatch

[René Fleschenberg]

Hi,

> Anyway as long as Django is installed in a virtualenv this shouldn't be
> too much of an issue, but I think we should expect some issues from the
> users and these should be documented otherwise people might get
> frustrated.

It can still be an issue for things that share the same virtualenv with 
Django, i.e. Django apps and projects. Any library that Django depends on 
imposes a restriction on which version of this library user code can use.

For example, if Django requires `requests>=2.9,<3.0`, this means that 
Django-based apps and projects cannot use requests 3.0+ in their code (at 
least not easily).

For this reason, I believe that Django should be very conservative about 
introducing new dependencies, but maybe not outrule them completely.

-- 
René

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1793582.LYRaI3gUmN%40rex.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[Donald Stufft]

> On Apr 7, 2016, at 1:31 AM, Sylvain Fankhauser  
> wrote:
> 
> The only problem I can see is dependencies conflicts, where Django would need 
> package X version > 1.0 and another installed package would need package X 
> version 0.9. I have this issue with a project of mine that required six >= 
> 1.10 and for some reason Xcode on OSX with the system Python (ie. not 
> installed with brew for example) forces you to have six 1.9, which comes 
> before in the PYTHONPATH. I know that Lektor  for 
> example solves this by having an install script 
>  that creates a virtualenv and installs 
> it inside this virtualenv.
> 
Yea. This is a problem with OS X Python from the system. They ship a bunch of 
libraries that are on the sys.path by default but which can’t be replaced (even 
more so in the most recent version of OS X when SIP is turned on which prevents 
installing anything into the system Python as well). This issue has been 
reported to Apple and I think there is a good chance that they’ll have it fixed 
in a way that a user can install packages into.

Overall though, users shouldn’t install things into their system Python which I 
think is supported by the fact that the system Python on recent versions of OS 
X are not writable, even with sudo, unless you disable one of the major 
security components of your machine. That doesn’t affect just any potential 
dependencies Django might have, but also Django itself.

> Also James mentioned that "back in the days pip wasn't as awesome as it is 
> today", but we have to keep in mind that a lot of OSes still use an old pip 
> version by default (Debian stable for example ships pip 1.5 
> ).
> 

pip 1.5 is new enough that you get most of the reliability benefits. Major 
things it’s missing are:

* Implicit wheel cache, allowing sdists to only be compiled once and the built 
binaries reused between installs.
* Implicit http cache, allowing files to only need to be downloaded once.
* Manylinux1 support, allowing people to publish binary wheels for Linux 
systems to PyPI.
* Conditional dependencies inside of a sdist without a hack workaround.

Minor, but nice quality of life things are:

* Massively reduced verbosity on install.
* Nicer progress bars.

> Anyway as long as Django is installed in a virtualenv this shouldn't be too 
> much of an issue, but I think we should expect some issues from the users and 
> these should be documented otherwise people might get frustrated.
> 
> Cheers,
> Sylvain
> 

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CCC99F08-1813-4E09-AE4B-B5E4BF58F618%40stufft.io.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Vendoring multipledispatch

[Donald Stufft]

> On Apr 7, 2016, at 8:20 AM, Erik Cederstrand  
> wrote:
> 
> 
>> Den 6. apr. 2016 kl. 13.42 skrev Marc Tamlyn :
>> 
>> Does anyone (potentially from OS packaging worlds maybe) have a good reason 
>> NOT to have a dependency?
> 
> Here is a list off the top of my head. This is not necessarily an argument 
> against dependencies, just some things to consider.
> 
> 
> 1: Availability. If Django depends on version x.y.z and x.y.z is removed from 
> PyPI, or the whole package is deleted, then Django is no longer installable 
> (google "NPM kik" for a recent example).
> 
> 2: Customization. We need to tweak functionality in some non-upstreamable 
> way, cherry-pick new functionality, or fix security issues before they are 
> published on PyPI.
> 
> 3: Version conflicts, as mentioned by Sylvain.
> 
> 4: Security/stability. We depend on version x.y and a witty developer uploads 
> dependency x.y.z+1 with an Easter egg, or the PyPI developer account is 
> hacked and x.y is replaced.
> 
> 
> These issues are amplified in a world where many people have automated 
> production deployments running 'pip install -U -r requirements.txt'. Issues 
> could spread very fast.
> 
> This may not be much different than what people are already exposed to with 
> their own project dependencies, but vendoring (directly or by dependency) is 
> endorsement by the Project, so any issues in the dependencies will fall back 
> on the Project.
> 


For 1, 3, and 4, Django is already exposed to these problems for the vast 
majority of use cases since if you want to use any database other than SQLite 
then you have to install something or if you want to use an ImageField, or 
bcrypt, etc. Having ``pip install Django`` work but not ``pip install Django 
psycopg2`` when you’re running a site that uses PostgreSQL doesn’t get you 
anything extra there.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/989F87C0-B9E1-4745-BCFC-ED19DC61C37F%40stufft.io.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Vendoring multipledispatch

[Erik Cederstrand]

> Den 6. apr. 2016 kl. 13.42 skrev Marc Tamlyn :
> 
> Does anyone (potentially from OS packaging worlds maybe) have a good reason 
> NOT to have a dependency?

Here is a list off the top of my head. This is not necessarily an argument 
against dependencies, just some things to consider.


1: Availability. If Django depends on version x.y.z and x.y.z is removed from 
PyPI, or the whole package is deleted, then Django is no longer installable 
(google "NPM kik" for a recent example).

2: Customization. We need to tweak functionality in some non-upstreamable way, 
cherry-pick new functionality, or fix security issues before they are published 
on PyPI.

3: Version conflicts, as mentioned by Sylvain.

4: Security/stability. We depend on version x.y and a witty developer uploads 
dependency x.y.z+1 with an Easter egg, or the PyPI developer account is hacked 
and x.y is replaced.


These issues are amplified in a world where many people have automated 
production deployments running 'pip install -U -r requirements.txt'. Issues 
could spread very fast.

This may not be much different than what people are already exposed to with 
their own project dependencies, but vendoring (directly or by dependency) is 
endorsement by the Project, so any issues in the dependencies will fall back on 
the Project.


Erik

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/19C24081-78F8-4790-9302-84BFB4AC0A46%40cederstrand.dk.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[Sylvain Fankhauser]

The only problem I can see is dependencies conflicts, where Django would
need package X version > 1.0 and another installed package would need
package X version 0.9. I have this issue with a project of mine that
required six >= 1.10 and for some reason Xcode on OSX with the system
Python (ie. not installed with brew for example) forces you to have six
1.9, which comes before in the PYTHONPATH. I know that Lektor
 for example solves this by having an install
script  that creates a virtualenv and
installs it inside this virtualenv. Also James mentioned that "back in the
days pip wasn't as awesome as it is today", but we have to keep in mind
that a lot of OSes still use an old pip version by default (Debian stable
for example ships pip 1.5
).


Anyway as long as Django is installed in a virtualenv this shouldn't be too
much of an issue, but I think we should expect some issues from the users
and these should be documented otherwise people might get frustrated.

Cheers,
Sylvain
On Apr 6, 2016 13:43, "Marc Tamlyn"  wrote:

> Does anyone (potentially from OS packaging worlds maybe) have a good
> reason NOT to have a dependency?
>
> In the event that Django does have Pip dependencies, I still think we
> should establish some rules. Things like they should be stable and mature,
> and perhaps widely known, and still be under active maintenance. These are
> all hazy ideas of course, but I think adding any new dependency should be
> something which we are cautious of, not something we do without careful
> consideration.
>
> If someone can put together some documentation or a DEP for how this
> process would work, I think this is a matter significant enough to talk
> about properly, and in a thread more clearly named. Such a proposal should
> also ideally reference what we do or have vendored before, why, and how
> well of otherwise that process has worked.
>
> Marc
> On 5 Apr 2016 9:49 a.m., "James Pic"  wrote:
>
>> Adding dependencies would definitely be a huge step forward. I think
>> Django doesn't have them because pip wasn't as awesome as it is today
>> back in the early days, but nowadays it would definitely make sense.
>> That would mean a bit more work for distribution package maintainers
>> but if we can start communicating with them about it then would there
>> be any reason not to do this move ?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CALC3Kae1Mt5LzYRq0PQbQcEMKwvbK%2Begr4FuC6baB5NZZtSbtA%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAMwjO1HqgJ%3DM93yWgmatZtc4s8%3DRB8FFfga24aJ3X_D_Ey2xdw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAAXOc44oMGNo38_CNw%2BnEKFEK_NWXPSPG8b0%2B1YKq%2Bi08ARMng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[Marc Tamlyn]

Does anyone (potentially from OS packaging worlds maybe) have a good reason
NOT to have a dependency?

In the event that Django does have Pip dependencies, I still think we
should establish some rules. Things like they should be stable and mature,
and perhaps widely known, and still be under active maintenance. These are
all hazy ideas of course, but I think adding any new dependency should be
something which we are cautious of, not something we do without careful
consideration.

If someone can put together some documentation or a DEP for how this
process would work, I think this is a matter significant enough to talk
about properly, and in a thread more clearly named. Such a proposal should
also ideally reference what we do or have vendored before, why, and how
well of otherwise that process has worked.

Marc
On 5 Apr 2016 9:49 a.m., "James Pic"  wrote:

> Adding dependencies would definitely be a huge step forward. I think
> Django doesn't have them because pip wasn't as awesome as it is today
> back in the early days, but nowadays it would definitely make sense.
> That would mean a bit more work for distribution package maintainers
> but if we can start communicating with them about it then would there
> be any reason not to do this move ?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CALC3Kae1Mt5LzYRq0PQbQcEMKwvbK%2Begr4FuC6baB5NZZtSbtA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMwjO1HqgJ%3DM93yWgmatZtc4s8%3DRB8FFfga24aJ3X_D_Ey2xdw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[James Pic]

Adding dependencies would definitely be a huge step forward. I think
Django doesn't have them because pip wasn't as awesome as it is today
back in the early days, but nowadays it would definitely make sense.
That would mean a bit more work for distribution package maintainers
but if we can start communicating with them about it then would there
be any reason not to do this move ?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALC3Kae1Mt5LzYRq0PQbQcEMKwvbK%2Begr4FuC6baB5NZZtSbtA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[Anssi Kääriäinen]

+1 to allowing dependencies.

For multipledispatch, I'd like to first verify the following items:
  1. Does the author consider the API for multipledispatch stable?
  2. Are the rules in multipledispatch what we want? If we vendor it,
we can easily change the matching rules.

On Tue, Apr 5, 2016 at 2:01 AM, Florian Apolloner  wrote:
>
>
> On Monday, April 4, 2016 at 3:58:09 PM UTC+2, Donald Stufft wrote:
>>
>> Without looking at this specific thing too closely, maybe it’s time for
>> Django to gain a required dependency instead of bundling or reinventing
>> everything?
>
>
> +I_do_not_know_how_much
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/9470fda1-67ab-4d92-b819-597d364fb460%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALMtK1H0LBvO4yKu35H67VK-LaSnizpF2h1xDYyGptf88W8Bdg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[Florian Apolloner]

On Monday, April 4, 2016 at 3:58:09 PM UTC+2, Donald Stufft wrote:
>
> Without looking at this specific thing too closely, maybe it’s time for 
> Django to gain a required dependency instead of bundling or reinventing 
> everything? 
>

+I_do_not_know_how_much 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9470fda1-67ab-4d92-b819-597d364fb460%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Vendoring multipledispatch

[Donald Stufft]

> On Apr 4, 2016, at 7:50 AM, Michał 'Khorne' Lowas-Rzechonek 
>  wrote:
> 
> So, I think I'm left with two options: reinvent 
> https://pypi.python.org/pypi/multipledispatch, or import that into 
> django.utils along with the license etc.
> 

Without looking at this specific thing too closely, maybe it’s time for Django 
to gain a required dependency instead of bundling or reinventing everything?


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3B506FDB-54D2-440D-8295-05DFDB88719D%40stufft.io.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Vendoring multipledispatch

[Michał 'Khorne' Lowas-Rzechonek]

Hi,

On last DjangoCon sprints I started working on ticket #26355 
, WIP is 
at https://github.com/django/django/pull/6395.

During the implementation I've realized that I need a double-dispatch 
mechanism for registering overloaded operators.
Current approach uses Python operator overloading exclusively, which can't 
(generally) take into account types of both operands.

So, I think I'm left with two options: 
reinvent https://pypi.python.org/pypi/multipledispatch, or import that into 
django.utils along with the license etc.

I appreciate this is not a decision to be taken lightly, so I'd rather ask 
here first, than submit a PR with shady imports.

Also, there might be a simpler/cleaner solution that doesn't required 3rd 
party code.

opinions?
-- 
Michał 'Khorne' Lowas-Rzechonek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4a8df55e-825c-4097-85bb-9b600d2c795a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.