Adding SASL authentication support to PyLibMCCache

[Ed Morley]

Hi

For apps running on Heroku, two of the main memcached options are
MemCachier and Memcached Cloud [1]. Both of these control access via SASL
authentication, which isn't supported by Django's current pylibmc backend
[2], even though pylibmc supports it [3]. As such, currently the
django-pylibmc backend has to be used instead [4].

I'd like to add SASL auth support to the Django pylibmc backend to make
this unnecessary, which is just a case of passing the relevant parameters
through to the pylibmc client during its instantiation [5]. A previous
newsgroup discussion about this is at [6].

SASL auth requires binary mode, so I'll also need to add support for that -
for which there is already a Django ticket filed [7].

I have a couple of questions:
a) Will a PR to add SASL authentication support be accepted?
b) If yes to (a), should the `username`, `password` and `binary` fields be
added to the top level `CACHES['foo']` dict, or nested inside `OPTIONS`
within that? I'm presuming within OPTIONS? Examples: [8].

Many thanks,

Ed

[1] https://elements.heroku.com/addons#caching
[2]
https://github.com/django/django/blob/1.9.1/django/core/cache/backends/memcached.py#L181
[3]
https://github.com/lericson/pylibmc/blob/1.5.0/src/pylibmc/client.py#L125
[4] https://github.com/django-pylibmc/django-pylibmc
[5] eg:
https://github.com/django-pylibmc/django-pylibmc/blob/v0.6.1/django_pylibmc/memcached.py#L85-L91
[6]
https://groups.google.com/forum/#!searchin/django-developers/pylibmc/django-developers/pISp20wuu0E/tamJ1h8zCzsJ
[7] https://code.djangoproject.com/ticket/15815
[8] https://emorley.pastebin.mozilla.org/8858134

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMtUAxsdbJu1TTk4KZiz9Zu%3DzC6-6WKEerS0Wv6gXgx_GzzTTQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Fellow Report - January 30, 2016

[Tim Graham]

Triaged

---

https://code.djangoproject.com/ticket/26130 - Convert ModelAdmin views to 
class-based views (duplicate)

https://code.djangoproject.com/ticket/26131 - model field choices are not 
enforced (invalid)

https://code.djangoproject.com/ticket/26134 - Update MySQL GIS function 
names to OpenGIS standard. (accepted)

https://code.djangoproject.com/ticket/26139 - UUIDField value must be 
declared "binary" to avoid MySQL warning (invalid)

https://code.djangoproject.com/ticket/26129 - bound forms do not display 
disabled field initial values (accepted)

https://code.djangoproject.com/ticket/26140 - BinaryField value must be 
declared "binary" in queries to avoid MySQL 5.7+ warning (created)

https://code.djangoproject.com/ticket/26133 - AbstractBaseUser missing 
password validaton (invalid)

https://code.djangoproject.com/ticket/26141 - __lt__ for model instances 
(wontfix)

https://code.djangoproject.com/ticket/26144 - Warn if a proxy model is an 
argument to dumpdata (accepted)

https://code.djangoproject.com/ticket/20415 - broken SRID in gis.admin 
javascript when "USE_THOUSAND_SEPARATOR = True" (fixed)

https://code.djangoproject.com/ticket/26145 - Made the debug context 
processor return queries for all databases (accepted)

https://code.djangoproject.com/ticket/26149 - Invalid migration generated 
when using order_with_respect_to and a unique_together constraint (accepted)

https://code.djangoproject.com/ticket/26152 - django.setup() hangs django 
if ever used in a module that's imported by an app (accepted)

https://code.djangoproject.com/ticket/26154 - Deprecate 
CommaSeparatedIntegerField (created)

https://code.djangoproject.com/ticket/26155 - Skip URL checks if 
settings.ROOTURL_CONF isn't defined (accepted)

Authored



https://github.com/django/django/pull/6034 - Fixed #26132 -- Discouraged 
use of TransactionTestCase.fixtures.

https://github.com/django/django/pull/6047 - Refs #26142 -- Documented that 
Formset's extra=0 doesn't prevent creating objects.

https://github.com/django/django/pull/6052 - Fixed #20415 -- Ensured srid 
isn't localized in OpenLayers JavaScript.

https://github.com/django/django/pull/6057 - Fixed #26137 -- Confirmed 
support for SpatiaLite 4.2/4.3 and added instructions.

https://github.com/django/django/pull/6058 - Fixed #26147 -- Adjusted some 
expected values in GIS tests.

https://github.com/django/django/pull/6061 - Fixed #26129 -- Made invalid 
forms display initial values of disabled fields.

https://github.com/django/djangoproject.com/pull/630 - Fixed #628 -- Made 
corporate membership logo required.

Reviewed/committed

--

https://github.com/django/django/pull/5735 - Fixed #25968 -- Changed 
project/app templates to use a "py-tpl" suffix.

https://github.com/django/django/pull/6028 - Fixed #26125 -- Fixed E731 
flake warnings.

https://github.com/django/django/pull/6003 - Fixed #26109 -- Raised a 
helpful error if loader.select_tamplate() is passed a string.git

https://github.com/django/django/pull/5984 - Fixed #25848 -- Set template 
origin on each node.

https://github.com/django/django/pull/6030 - Fixed #26122 -- Fixed copying 
a LazyObject

https://github.com/django/django/pull/6002 - Fixed #25546 -- Prevented 
duplicate queries with nested prefetch_related().

https://github.com/django/django/pull/5886 - Fixed #25354 -- Added 
class/app_label interpolation for related_query_name.

https://github.com/django/django/pull/6033 - Refs #26015 -- Added 
regression test for an empty sitemap.

https://github.com/django/django/pull/6039 - Fixed #26120 -- Made 
HStoreField cast keys & values to strings.

https://github.com/django/django/pull/5995 - Fixed #26022 -- Cleaned up 
assertRaises* test assertions.

https://github.com/django/django/pull/6064 - Fixed #14402 -- Removed 
clearing of help_text for ManyToManyField's raw_id_fields.

Reviews of core dev work


https://github.com/django/django/pull/6029 - Fixed #26135 -- Fixed the 
migration questioner's handling of disabled apps.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/27e2ed80-412d-4315-874a-ec685def7974%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Extending ModelForm.save_m2m

[James Pic]

Hi all,

Many apps provide new related managers to extend your django models with.
For example, django-tagulous provides a TagField which abstracts an M2M
relation with the Tag model, django-gm2m provides a GM2MField which
abstracts an relation, django-taggit provides a TaggableManager which
abstracts a relation too, django-generic-m2m provides
RelatedObjectsDescriptor which abstracts a relation again.

While that works pretty well, it gets a bit complicated when it comes to
encapsulating the business logic for saving such data in a form object.
This is two-part problem:

- getting initial data,
- saving relations.

Currently in Django, getting initial data is done in model_to_dict, which
call's the model field's value_from_object() method, bypassing form fields
completely, I suggest that we add ability to do that in the Form field too.

As for saving relations, this is currently done in ModelForm.save_m2m(),
which calls the model field's save_form_data() directly, for each field
that's many to many or virtual, completely ignoring the form field.

This means that if one wants to customize how a form field works in terms
of getting the initial data for its widget and saving relations, then they
should override the model field, instead of just the form field. What could
go wrong with that ? :)

Do you think it would be OK to decouple that a bit and allow the form field
to have value_from_object() and save_form_data() ?

Best

James

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALC3KadPbQ0xCLAZduQGjmAeiirjaaqOu5%3DP_Spu5K1TSPN3yQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improving MSSQL and Azure SQL support on Django

[Tim Allen]

Hi Cristiano, I support a bunch of developers on Windows (and Mac and 
Linux). You've probably heard this before, but Vagrant has really been a 
game changer for us. We're a RedHat shop, so I've built a CentOS vagrant 
box our developers can easily spin up for development purposes. It really 
helps for several reasons: they're developing on a Linux flavor as close to 
the production environment as possible, and have root access on their local 
VM which allows them to test new packages, configurations, and so on. It 
has improved our development process by leaps and bounds. For my stuff, 
I've developed a Vagrant / Linux / Python 3 / Django / FreeTDS - SQL Server 
box. I don't know if it might help you out, but feel free to have a look 
here: https://github.com/FlipperPA/django-python3-vagrant

I'm still hoping this project moves forward; it'd be great to have 
closer-to-native support.

Regards,

Tim

On Thursday, January 28, 2016 at 1:09:41 PM UTC-5, Cristiano Coelho wrote:
>
> Tim Allen,
>
> What you said about compiling the C dependencies on a similar machine and 
> then upload it all together indeed works (it was one of the options) but 
> caused some other issues (ie: we usually develop on Windows, and also the 
> compiled libraries are very platform specific) and performance was really 
> not that important in this case. But just letting you know that your idea 
> works most of the time if you are willing to take the extra work.
>
> El jueves, 28 de enero de 2016, 12:48:29 (UTC-3), Tim Allen escribió:
>>
>> Thanks to everyone for their efforts; my workplace has a mix of SQL 
>> Server and PostgreSQL, heavier on the SQL Server side. Due to some groups 
>> reliance on SSIS and tight SQL Server integration with data vendors, that 
>> isn't going to change any time soon, so this is project is one we're 
>> following closely as well. We've tried to contribute by way of feedback, 
>> testing various configurations with various drivers, some documentation and 
>> a minuscule amount of code contribution.
>>
>> In case this anecdotal evidence helps anyone in the meantime, the stack 
>> we've found most reliable these days (from RedHat / CentOS, at least, but 
>> also partially tested on Ubuntu) is:
>>
>> - FreeTDS 0.95 (supports TDS version 7.3) with unixODBC. We tried the 
>> Microsoft provided ODBC driver, but ran into quite a few issues, 
>> particularly with multi-threading.
>> - pyodbc 3.0.10. pyodbc just works. We get slightly better performance 
>> with pymssql, but have found pyodbc to be more frequently updated and 
>> rock-solid. The performance upgrade didn't warrant using pymssql in our 
>> case, but is worth mentioning.
>> - django-pyodbc-azure. This is kept up to date with Django and Python 
>> release versions, and works with the least amount of configuration tweaking 
>> that we have found.
>>
>> We're on a mix of RHEL/CentOS 6 and 7, and have gotten this stack running 
>> reliably up to v7.2. YMMV, of course!
>>
>> As for the C dependencies, have you considered building the C binaries 
>> necessary on another server, and then just including them as part of a 
>> wheel (or something like that) for installation were you couldn't install? 
>> This sound like a perfect use case for a temporary vagrant box you could 
>> blow away after compiling. Just a thought that might give you the 
>> performance you need without stepping on anyone's toes.
>>
>> On Wednesday, January 27, 2016 at 12:15:48 AM UTC-5, Cristiano Coelho 
>> wrote:
>>>
>>> I'm interested in the progress of this as well :)
>>>
>>> Sorry I didn't read through all the posts, mostly the first ones about 
>>> the idea.
>>>
>>> I would like to know, have you guys decided on which adapter to use? I 
>>> have had a project where we needed to connect to SQL Server from a linux 
>>> machine (actually amazon lambda) and even worse, we couldn't install any 
>>> library with dependencies on C code, so we used one that was implemented in 
>>> pure python that worked very well (pytds if I'm not wrong), with ofcourse, 
>>> not the best performance.
>>> Why do I tell this? Because even if you want django to run on SQL 
>>> Server, it doesn't really mean you want to run it on a Windows machine, 
>>> actually, that would probably be a terrible idea (no ofense), since apache 
>>> works horribly bad on Windows, and Linux is atually the best OS to run a 
>>> web server with python code (either nginx or apache). So please keep this 
>>> in mind when chosing a connector, since if it has C dependencies (which it 
>>> will probably have, since the pure python ones are quite slow).
>>>
>>> About if you need different connectors for Azure or SQLServer, I'm 
>>> 'almost' sure you don't, we use azure or other cloud based sqlserver 
>>> deployments with no problem with standard sqlserver connectors.
>>>
>>> So basically, do not aim this towards making django more Windows 
>>> friendly, but rather the actual SQL Server backend.
>>>
>>> El lunes, 25 de enero de 

Re: Deprecate Cache.has_key()?

[Tim Graham]

Yes, but dict.has_key() is removed in Python 3. If we started the 
deprecation in Django now, it would be scheduled for removal in Django 2.0 
which also drops support for Python 2.

On Friday, January 29, 2016 at 8:04:33 PM UTC-5, Curtis Maloney wrote:
>
> Isn't this so it complies with the dict interface? True, it's discouraged 
> now, but it's still there.
>
>
>
> On 30 January 2016 10:42:12 am LHDT, Tim Graham  > wrote:
>>
>> Also the template BaseContext class has an undocumented has_key() method. 
>> I wonder if that can be removed without a deprecation?
>>
>> https://github.com/django/django/pull/6066
>>
>> On Friday, January 29, 2016 at 6:11:54 PM UTC-5, Tim Graham wrote:
>>>
>>> Any opposition to deprecating Cache.has_key()? Cache already implements 
>>> __contains__() so converting has_key() to "in" can be done with existing 
>>> versions of Django. Anyone using flake8 will get a warning about has_key() 
>>> usage: W601 .has_key() is deprecated, use 'in'.
>>>
>>
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9aebfff7-b202-4bae-87de-c35796333587%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Replacing the contrib.sites Site model with a setting?

[Aymeric Augustin]

This is a valid use case, but not one django.contrib.sites (officially) 
supports, since it requires a constant SITE_ID setting.

While Tim's suggestion doesn't cause a regression, it doesn't make this pattern 
easier to implement either.

It's definitely worth considering.

-- 
Aymeric.

> Le 30 janv. 2016 à 08:01, Max Arnold  a écrit :
> 
> What if this list of sites needs to be changed dynamically without app 
> restart (multitenancy)?
> 
>> On Fri, Jan 29, 2016 at 12:45:02PM -0800, Tim Graham wrote:
>> In another thread about adding a "scheme" field to the Site model [1], I 
>> floated the idea of moving the data stored by the Site model into a setting:
>> 
>> I've sometimes thought that the Site model violates the principle that you 
>> shouldn't put configuration in your database. I guess there's some 
>> usefulness to having a ForeignKey to the site, but... would it be feasible 
>> to offer a SITES setting that could be used instead? e.g.
>> 
>> SITES = {
>>1: {'scheme': 'http', 'domain': example.com, 'name': 'My Site'},
>>...
>> }
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/20160130070141.GA2770%40otg.dm.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BD11589E-99B7-4D14-9F60-5D346D557B98%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.