Re: skipping elidable migrations

[Dan Watson]

Hi Adam,

All of your suggestions are perfectly valid things to do (and are in fact 
things I've done in the past). This particular project uses data migrations 
fairly heavily, and the developers haven't been particularly good about 
making sure they run cleanly on fresh databases. We are using Django's test 
framework, but a) with migrations disabled for performance reasons and b) 
using --keepdb, which masks these types of problems.

It just got me thinking that if you've gone out of your way to mark an 
operation as basically "safe to delete", it seems unimportant to run them 
during an initial migration. Bad programming aside, it seems like an easy 
performance bump for projects that use these types of operations. I think 
of elidable as a "soft delete" option that gets garbage collected next 
squash. This would just be an optimization step to ignore what is already 
marked for death. Maybe it's not an optimization we want to reason about, 
or maybe it isn't a big enough use case to be worth coding for, but I 
figured I'd put it out there.

Regards,
Dan


On Saturday, November 10, 2018 at 8:13:27 AM UTC-5, Adam Johnson wrote:
>
> Do you have an example? If you're using Django's default testing 
> framework, it normally creates a fresh database, so from the moment a new 
> data migration is written it would be tested with the empty database 
> scenario.
>
> Afaiu it is possible to write RunPython operations in a way that no-ops on 
> empty databases, e.g. starting with the equivalent of "if not 
> Model.object.exists(): return." It is also possible with RunSQL on database 
> backends that support conditional logic around SQL statements,  or they can 
> be easily converted to RunPython using a function that uses 
> cursor.execute(). And if a project develops problems with past migrations, 
> it's always possible to (carefully) edit them to include such conditions.
>
> On Fri, 9 Nov 2018 at 16:11, Dan Watson > 
> wrote:
>
>> Hi All,
>>
>> I was wondering if anyone had any thoughts on an option to the "migrate" 
>> command (--skip-elidable?) that would skip running elidable migrations. The 
>> use case here is that data migrations that build up over time may act on 
>> certain assumptions (existing tables/data) that may not be true when 
>> migrating a new database. It seems that since they were explicitly marked 
>> as able to be deleted when squashing, they would be safe to not run when 
>> creating a new database. Maybe we don't go so far as to make this the 
>> default behavior when migrating a fresh database, but an option would be 
>> nice. I realize you could simply squash your migrations, but that's not 
>> without penalty of code churn, testing, etc. especially when your existing 
>> migration graph is otherwise fine (and performant).
>>
>> If there's some consensus about this being worthwhile, or at least no 
>> strong objections to it, I can take a stab at the implementation.
>>
>> Regards,
>> Dan
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/fba7a58a-444b-4c90-b139-151580423366%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/fba7a58a-444b-4c90-b139-151580423366%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9cbccecc-0831-4c62-869d-ba7ca619ded7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

skipping elidable migrations

[Dan Watson]

Hi All,

I was wondering if anyone had any thoughts on an option to the "migrate" 
command (--skip-elidable?) that would skip running elidable migrations. The 
use case here is that data migrations that build up over time may act on 
certain assumptions (existing tables/data) that may not be true when 
migrating a new database. It seems that since they were explicitly marked 
as able to be deleted when squashing, they would be safe to not run when 
creating a new database. Maybe we don't go so far as to make this the 
default behavior when migrating a fresh database, but an option would be 
nice. I realize you could simply squash your migrations, but that's not 
without penalty of code churn, testing, etc. especially when your existing 
migration graph is otherwise fine (and performant).

If there's some consensus about this being worthwhile, or at least no 
strong objections to it, I can take a stab at the implementation.

Regards,
Dan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fba7a58a-444b-4c90-b139-151580423366%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: about ticket 28588- has_perm hide non-existent permissions

[Dan Watson]

Seems like maybe it would be more helpful if has_perm logged a note about 
the permission not existing (probably only in debug), rather than just 
returning False. In fact, I'd argue it should still return True -- if the 
permission did exist, the superuser would have it. And there's a 
backwards-compatibility argument. Think of superusers more as "permissions 
don't apply to me" than "I have all permissions".

Dan

On Sunday, September 24, 2017 at 10:56:40 AM UTC-4, moshe nahmias wrote:
>
> Hi,
> I am a python developer and like to use Django for web development.
> Since I like the framework I want to contribute back, so I looked at the 
> open tickets to find something I can start with contributing and found 
> ticket 28588.
>
> This ticket is about when checking if the user has permission for some 
> action if the user is super user he/she gets it all the time, even when the 
> permission doesn't exist, and this is not developer friendly because the 
> developer can mistakenly think that everything is fine even when the 
> permission doesn't exist.
>
> As I understand (and correct me if I'm wrong) there should be a discussion 
> about if we want to do this.
>
> If accepted I would like to do this, I think it's an easy enough change 
> for a new contributor like me.
>
> As I understand the ticket the problem is that a developer gets confused 
> on this behaviour (and it's illogical) that the super user is having a 
> permission that doesn't exist.
>
> What do you think? (I think I will discuss my solution or optional 
> solutions after we decide if we want to change this behaviour)
>
> [1] https://code.djangoproject.com/ticket/28588
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d0430a0f-04d9-484d-b888-bddf6f53ff95%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Suggested documentation: PG server-side cursors used in .iterator() only work inside a transaction

[Dan Watson]

Declaring a cursor WITH HOLD means it can be used outside transactions, but 
it seems like the server-side cursors are already set up WITH HOLD when 
autocommit is on:

https://github.com/django/django/blob/master/django/db/backends/postgresql/base.py#L212

So I'm guessing maybe you've disabled transaction management entirely 
(AUTOCOMMIT=False)? In that case, you could try enabling autocommit 
manually when you need to use .iterator(), but maybe the backend could 
check "connection.autocommit or not connection.in_atomic_block" when 
setting WITH HOLD, so even if autocommit is off, WITH HOLD would be used if 
not using an atomic block?

As for the behavior when trying to iterate a cursor after the transaction 
has been exited, I'd expect an error. The cursor should be destroyed at the 
end of the transaction (unless WITH HOLD was used, which it shouldn't be 
inside a transaction), so trying to FETCH from it should raise an error.

Dan

On Sunday, August 6, 2017 at 10:34:35 PM UTC-4, Josh Smeaton wrote:
>
> Yes we should be documenting edge cases and unexpected results. We have a 
> page that discusses some issues with server side cursors: 
> https://docs.djangoproject.com/en/dev/ref/databases/#transaction-pooling-server-side-cursors
>
> Is there anyway we could make SSC work without a transaction? We'd prefer 
> to fix than document if possible.
>
> On Thursday, 3 August 2017 07:22:23 UTC+10, Evan Heidtmann wrote:
>>
>> Hey all,
>>
>> The docs for qs.iterator() say that, in Django 1.11 on Postgres, a 
>> server-side cursor is used.
>>
>> Oracle and PostgreSQL 
>>> 
>>>  use 
>>> server-side cursors to stream results from the database without loading the 
>>> entire result set into memory.
>>
>>
>> https://docs.djangoproject.com/en/1.11/ref/models/querysets/#iterator
>>
>> I discovered that this is only true if the query is run inside a 
>> transaction. Outside a transaction, it appears that Django falls back to a 
>> regular SELECT query, which could be extremely expensive and is certainly 
>> unexpected.
>>
>> I don't know what happens if you call .iterator() inside a transaction 
>> block and then exit that block. Undefined behavior?
>>
>> Therefore I suggest the following docs edit (changes in italic):
>>
>> Oracle and PostgreSQL 
>>> 
>>>  use 
>>> server-side cursors to stream results from the database without loading the 
>>> entire result set into memory. *On PostgreSQL, server-side cursors are 
>>> used only if .iterator() is called within a transaction.*
>>
>>
>> And thanks for this new feature -- it's transformative for my workload.
>>
>> -Evan
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/89ff1578-93d6-4146-b0a5-792344e64000%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using AbstractBaseUser without django.contrib.auth

[Dan Watson]

On Thursday, April 2, 2015 at 10:38:48 AM UTC-4, Marc Tamlyn wrote:
>
> Apologies, I was confusing abstract base user and abstract user there. 
> Seems your proposal should work. Have you opened a ticket?
>

I have:

https://code.djangoproject.com/ticket/24564
https://github.com/django/django/pull/4438

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a46911df-7bfa-477c-a2cd-3e5b0462880f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using AbstractBaseUser without django.contrib.auth

[Dan Watson]

On Thursday, April 2, 2015 at 2:27:51 AM UTC-4, Marc Tamlyn wrote:
>
> Moving them into another module won't make much difference as their 
> definition requires Permission and Group and therefore they'd still need to 
> import Permission and Group. We'd need an "AbstractAbstractBaseUser" which 
> would be silly.
>
>
 AbstractBaseUser doesn't require any models from contrib.auth (as opposed 
to AbstractUser, which inherits PermissionsMixin), and I'm fine with 
leaving PermissionsMixin in models.py since it does require Group and 
Permission.

Of course, there is no requirement to use AbstractBaseUser for your custom 
> model at all, though this does result in some otherwise unnecessary code 
> duplication. I would say your choice is either two empty tables in your 
> database, or code duplication between Django's source and your custom user. 
> Personally I'd prefer the former.
>

Very true, but it seems like AbstractBaseUser was designed to be used by 
systems not wanting Django's permission structure. It's even documented[1] 
how to make AbstractBaseUser subclasses work with the admin. I can live 
with either wart (copying code or empty tables) if need be, just wanted to 
explore some alternatives. I'll open a ticket.

[1] 
https://docs.djangoproject.com/en/1.8/topics/auth/customizing/#custom-users-and-django-contrib-admin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/297c3eb4-7832-424a-a269-4af471eae23a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using AbstractBaseUser without django.contrib.auth

[Dan Watson]

On Wednesday, April 1, 2015 at 10:50:17 PM UTC-4, Curtis Maloney wrote:
>
> Does your model inherit from PermissionsMixin?
>
> If you're using Admin, or any of Django's permissions machinery, you will 
> need django.contrib.auth in there to use the Group and Permission models.
>
>
I'm using my own permissions system, but implementing 
everything PermissionsMixin does (without subclassing it) so the admin 
still works.
 

> --
> Curtis
>
> On 2 April 2015 at 13:47, Dan Watson > 
> wrote:
>
>> While trying out Django 1.8 with one of my sites that uses a custom User 
>> model (and doesn't have django.contrib.auth in INSTALLED_APPS), I noticed a 
>> few unexpected deprecation warnings:
>>
>> /Users/dcwatson/Documents/Environments/reader/lib/python3.4/site-packages/django/contrib/auth/models.py:41:
>>  
>> RemovedInDjango19Warning: Model class django.contrib.auth.models.Permission 
>> doesn't declare an explicit app_label and either isn't in an application in 
>> INSTALLED_APPS or else was imported before its application was loaded. This 
>> will no longer be supported in Django 1.9.
>>   class Permission(models.Model):
>>
>> Same thing for User and Group. Is it required to have django.contrib.auth 
>> installed to use custom User models now? Seems like it would be easy enough 
>> to move AbstractBaseUser (and probably UserManager/BaseUserManager) to a 
>> separate modules and import them from django.contrib.auth.models, so sites 
>> could use them without triggering these errors. If this sounds reasonable, 
>> I can open a ticket/PR.
>>
>> Dan
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/02f98a6f-9fbf-4c57-8285-80f3c2c3e2ef%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/02f98a6f-9fbf-4c57-8285-80f3c2c3e2ef%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/44df83a3-5006-4be0-bbb1-cb5d0803381d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Using AbstractBaseUser without django.contrib.auth

[Dan Watson]

While trying out Django 1.8 with one of my sites that uses a custom User 
model (and doesn't have django.contrib.auth in INSTALLED_APPS), I noticed a 
few unexpected deprecation warnings:

/Users/dcwatson/Documents/Environments/reader/lib/python3.4/site-packages/django/contrib/auth/models.py:41:
 
RemovedInDjango19Warning: Model class django.contrib.auth.models.Permission 
doesn't declare an explicit app_label and either isn't in an application in 
INSTALLED_APPS or else was imported before its application was loaded. This 
will no longer be supported in Django 1.9.
  class Permission(models.Model):

Same thing for User and Group. Is it required to have django.contrib.auth 
installed to use custom User models now? Seems like it would be easy enough 
to move AbstractBaseUser (and probably UserManager/BaseUserManager) to a 
separate modules and import them from django.contrib.auth.models, so sites 
could use them without triggering these errors. If this sounds reasonable, 
I can open a ticket/PR.

Dan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/02f98a6f-9fbf-4c57-8285-80f3c2c3e2ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: unique by default on SlugFields

[Dan Watson]

On Wednesday, June 27, 2012 7:15:03 AM UTC-4, Meshy wrote:
>
> Perhaps it's just me, but I've very rarely wanted a SlugField that wasn't 
> unique. Would this not be a sensible default? I realise that a lot of apps 
> will rely upon this default, but objectively speaking would this not be 
> better? Perhaps this change would be appropriate for django 2.0.
>
> At the moment, slug = models.SlugField()creates a non-unique field, and 
> if you want it to be unique, then you must add unique=True. I feel this 
> is wrong.
>
> It seems to me that unique should be default, and if you don't want a 
> unique slug, you should explicitly state that:
> slug = models.SlugField(unique=False)
>

I'd agree that slugs are often unique, but I think it's just as common that 
they are unique together with another field (think blogs -- 
/blog/2012/06/27/some-slug/ -- where the slug is unique_together with the 
date). For this case, you'd have to remember to specify unique=False *and* 
unique_together.
 

> I've added an issue on the tracker for this: 
> https://code.djangoproject.com/ticket/18525
>
> I realise this may be a contentious issue...or that I may even get shot 
> down in flames on this one ;P Go easy on me! :)
>
> What do you all think?
>

It doesn't seem like a burden to have to say unique=True if you truly want 
a singularly unique slug. Making it explicit, aside from being good python 
zen, also helps to see the natural keys of your model at a glance.

Dan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/8V3D_HcVhf8J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: revisiting #7726

[Dan Watson]

On Saturday, July 30, 2011 8:59:48 PM UTC-4, Russell Keith-Magee wrote:
>
> On Fri, Jul 29, 2011 at 11:07 PM, Dan Watson  wrote:
> > We were recently bitten by a backwards-incompatible change that seemed to
> > creep in between 1.2.4 and 1.2.5. DecimalFields that define max_digits ==
> > decimal_places used to be valid until #7726 [1]. When trying to upgrade 
> to
> > 1.2.5, several models failed validation. Changing them such that
> > max_digits=decimal_places+1 worked around the problem, but means we need 
> to
> > take an extra validation step to ensure the value is < 1. As Karen Tracey
> > mentioned in #7064 [2], numeric(x,x) is perfectly valid at the database
> > level, so it seems strange that Django would not support this. It seems 
> like
> > what may have happened was:
> >   1. #7064 was opened
> >   2. #7726 was opened before #7064 was fixed, i.e. you could not
> > successfully save a decimal field at the time, since the normalized
> > representation has a leading 0.
> >   3. #7064 was fixed (in my opinion, obviating the need for #7726)
> >   4. #7726 was fixed, essentially overriding #7064
> > Unless I'm misunderstanding something, I'd propose rolling back #7726, 
> since
> > #7064 made it possible to use (and create) decimal fields with max_digits 
> ==
> > decimal_places.
> > Regards,
> > Dan
> >
> > [1] https://code.djangoproject.com/ticket/7726
> > [2] https://code.djangoproject.com/ticket/7064
>
> Rolling back #7726 isn't the right fix here, because it catches a
> legitimate validation case (decimal places > max_digits). However, the
> check is a little overenthusiastic. From a quick inspection, I think
> modifying the validation condition on line 77 to be a > rather than >=
> should do the trick.
>
> This should be opened as a separate ticket (tracking the "max_digits
> == decimal_places is legal, but raises validation error" problem);
> given that it's a regression from 1.2.4, it should also be marked as a
> release blocker.
>
> Yours,
> Russ Magee %-)
>

Thanks for the feedback, here's the ticket with a patch:

https://code.djangoproject.com/ticket/16570

Dan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/wdS36vwm0B4J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

revisiting #7726

[Dan Watson]

We were recently bitten by a backwards-incompatible change that seemed to 
creep in between 1.2.4 and 1.2.5. DecimalFields that define max_digits == 
decimal_places used to be valid until #7726 [1]. When trying to upgrade to 
1.2.5, several models failed validation. Changing them such that 
max_digits=decimal_places+1 worked around the problem, but means we need to 
take an extra validation step to ensure the value is < 1. As Karen Tracey 
mentioned in #7064 [2], numeric(x,x) is perfectly valid at the database 
level, so it seems strange that Django would not support this. It seems like 
what may have happened was:

  1. #7064 was opened
  2. #7726 was opened before #7064 was fixed, i.e. you could not 
successfully save a decimal field at the time, since the normalized 
representation has a leading 0.
  3. #7064 was fixed (in my opinion, obviating the need for #7726)
  4. #7726 was fixed, essentially overriding #7064

Unless I'm misunderstanding something, I'd propose rolling back #7726, since 
#7064 made it possible to use (and create) decimal fields with max_digits == 
decimal_places.

Regards,
Dan

[1] https://code.djangoproject.com/ticket/7726
[2] https://code.djangoproject.com/ticket/7064

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/SxicWAsLc4gJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: combining querysets with isnull

[Dan Watson]

On Thursday, October 14, 2010 7:03:35 PM UTC-4, Johannes Dollinger wrote:
>
>
> Am 14.10.2010 um 23:07 schrieb Dan Watson:
>
> > There seems to be some inconsistent behavior when combining querysets
> > that use isnull on a ForeignKey. I'm not sure how to explain it well
> > in plain english, so here's a boiled-down example:
> > 
> > # Models
> > 
> > class Item (models.Model):
> >title = models.CharField( max_length=100 )
> > 
> > class PropertyValue (models.Model):
> >label = models.CharField( max_length=100 )
> > 
> > class Property (models.Model):
> >item = models.ForeignKey( Item, related_name='props' )
> >key = models.CharField( max_length=100 )
> >value = models.ForeignKey( PropertyValue, null=True )
> > 
> > # Example
> > 
> > item = Item.objects.create(title='Some Item')
> > pv = PropertyValue.objects.create(label='Some Value')
> > item.props.create(key='a', value=pv)
> > item.props.create(key='b')
> > q1 = Q(props__key='a', props__value=pv)
> > q2 = Q(props__key='b', props__value__isnull=True)
> > qs1 = Item.objects.filter(q1) & Item.objects.filter(q2)
> > qs2 = Item.objects.filter(q2) & Item.objects.filter(q1)
> > 
> > I would have expected qs1 and qs2 to yield the same result, but they
> > do not. They should both return the single item, but qs1 returns
> > nothing. The SQL for qs1 looks like:
> > 
> > SELECT "app_item"."id", "app_item"."title" FROM "app_item"
> > INNER JOIN "app_property" ON ("app_item"."id" =
> > "app_property"."item_id")
> > LEFT OUTER JOIN "app_property" T4 ON ("app_item"."id" = T4."item_id")
> > LEFT OUTER JOIN "app_propertyvalue" T5 ON ("app_property"."value_id" =
> > T5."id")
> > WHERE (("app_property"."value_id" = 1  AND "app_property"."key" =
> > 'a' ) AND (T5."id" IS NULL AND T4."key" = 'b' ))
> > 
> > The first app_property join corresponds to q1, and the second
> > corresponds to q2. However, the app_propertyvalue join (corresponding
> > to the isnull from q2) refers to app_property.value_id (i.e. q1)
> > instead of T4.value_id (i.e. q2). I think fixing #10790 would fix
> > this, since removing the join to app_propertyvalue and simply checking
> > "T4.value_id IS NULL" works as expected, but I'm not sure if this is a
> > separate problem worthy of another ticket or not. Also (less
> > importantly), since both criteria (q1 and q2) are checking
> > props__key='something', I would imagine both joins could be INNER
> > JOINs, but the isnull seems to promote the second to a LEFT OUTER
> > JOIN.
> > 
> > I guess I was just wondering if I should open a separate ticket for
> > this, or is this simply a symptom of #10790? Or am I missing
> > something?
>
> Your problem looks like #11052 [1].
>
> [1] http://code.djangoproject.com/ticket/11052
>
> __
> Johannes
>

Looking at this further, I think the problem is slightly different. #11052 
[1] describes a situation where joins are not promoted where they should be. 
The attached patch also did not produce correct results for my test case. I 
think #10790 [2] is more closely related, but still isn't exactly the issue 
(and again, the attached patch does not fix my test case). The issue I'm 
seeing is that, when combining queries, the join conditions do not reference 
aliases from the "correct" side in all cases. In my example above, the join 
to T5 should have referenced T4 since they both came from the right side of 
the combination. However, the combine method does not check to see if the 
left side of the join condition (for tables on the right) has already been 
re-aliased. I've opened ticket #15823 [3] with a patch that passes all 
tests, and a new regression test. As these internals get pretty hairy, I'd 
love if someone could take a look and make sure this is the correct 
approach.

Dan

[1] http://code.djangoproject.com/ticket/11052
[2] http://code.djangoproject.com/ticket/10790
[3] http://code.djangoproject.com/ticket/15823

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14733: A vote in favor of no validation of .raw() queries

[Dan Watson]

On Wednesday, March 9, 2011 4:15:43 PM UTC-5, Xof wrote:
>
> Hi,
>
> I'd like to offer a vote in favor of accepting the original patch to 
> #14733, which removes the validation of the query done in a .raw() operation 
> on a QuerySet.
>
> The current situation is that Django requires that any query passed in 
> begin with the literal string "SELECT", under the theory that only things 
> beginning with SELECT return results ("set-returning operations").  This 
> isn't correct.
>
> In PostgreSQL, as it stands right now, operations which return sets can 
> begin with:
>
> SELECT
> FETCH
> INSERT
> WITH
> TABLE
>
> (I may have missed some.)
>
> This list isn't static, either; DO might well return sets in the future, 
> although it doesn't right now.  And, of course, the exact list of the things 
> that can return sets is backend-specific; that's just PG's list.
>
> Given that .raw() is very much a "You must know what you are doing" feature 
> in the first place, I don't see the need to be strict about the input, at 
> the cost of some very useful functionality.
>

I very much agree. I'd love to be able to FETCH from server-side cursors 
using .raw().

Dan 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Re-open #7231: New "join" parameter for the "extra" QuerySet method

[Dan Watson]

On Tuesday, March 8, 2011 6:16:26 PM UTC-5, Russell Keith-Magee wrote:
>
> On Wed, Mar 9, 2011 at 7:09 AM, Jacob Kaplan-Moss  
> wrote:
> > On Tue, Mar 8, 2011 at 9:35 PM, bendavis78  wrote:
> >> I'd like to start a discussion on this since russelm closed the
> >> issue.  There are a few other people that believe the issue should be
> >> left open.   I've been using this patch for nearly two years, and have
> >> found it to be useful in several different cases.  I disagree that
> >> the  .raw() functionality is a sufficient alternative, as it is not
> >> possible to modify an existing queryset using .raw().  For example,
> >> if I have a function that accepts a queryset, I want to be able to
> >> modify that queryset by giving it a extra info for the JOIN and SELECT
> >> clauses.
> >
> > .extra() was a kludge that existed because .raw() didn't. Frankly, I'm
> > considering deprecating and removing .extra() entirely:
>
> Yes. Yes Yes Yes. Yes. Oh, and Yes. +1.
>
> > I've rarely
> > seen a case where using it didn't come back to cause problems in the
> > future. I'm certainly going to be a strong -1 on adding any more
> > "features" to .extra().
>
> Agreed. From an engineering perspective, extra() is the single most
> fragile part of the ORM. Dumping extra would make me extraordinarily
> happy.
>
It's also incredibly useful for certain situations. Two that come 
immediately to mind are augmenting objects returned with an extra field via 
custom SQL (a subselect for instance), and using custom SQL (e.g. to_tsquery 
for full-text searches in Postgres) in the WHERE clause. Both *could* be 
accomplished using .raw() if you don't mind giving up .filter(), 
.order_by(), etc. But it would be a shame to throw out everything a queryset 
gives you just because you need hooks into the SQL it generates.

If you're seriously considering deprecating .extra(), I think it would be 
beneficial to put together some use cases (such as those above) that could 
be covered in a less-fragile way.

Dan

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Problem with ``Model.objects.create``

[Dan Watson]

On Wednesday, January 5, 2011 3:30:42 AM UTC-5, Valts wrote:
>
> Just my $0.02: I think that such feature would be useful when using 
> get_or_create() when I don't want the object to be saved into the 
> database if it does not validate. There could be added another special 
> keyword argument "full_clean", "validate" or similar to 
> get_or_create(). If the proposed idea gets the green light I am 
> willing to create the ticket and the necessary patches. 
>

While I tend to agree that it would be nice to have an option to validate 
models before saving automatically, it's super easy to add yourself using 
the pre_save signal:

from django.db.models.signals import pre_save
def validate_model(sender, **kwargs):
if 'raw' in kwargs and not kwargs['raw']:
kwargs['instance'].full_clean()
pre_save.connect(validate_model, dispatch_uid='validate_models')

Dan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

combining querysets with isnull

[Dan Watson]

There seems to be some inconsistent behavior when combining querysets
that use isnull on a ForeignKey. I'm not sure how to explain it well
in plain english, so here's a boiled-down example:

# Models

class Item (models.Model):
title = models.CharField( max_length=100 )

class PropertyValue (models.Model):
label = models.CharField( max_length=100 )

class Property (models.Model):
item = models.ForeignKey( Item, related_name='props' )
key = models.CharField( max_length=100 )
value = models.ForeignKey( PropertyValue, null=True )

# Example

item = Item.objects.create(title='Some Item')
pv = PropertyValue.objects.create(label='Some Value')
item.props.create(key='a', value=pv)
item.props.create(key='b')
q1 = Q(props__key='a', props__value=pv)
q2 = Q(props__key='b', props__value__isnull=True)
qs1 = Item.objects.filter(q1) & Item.objects.filter(q2)
qs2 = Item.objects.filter(q2) & Item.objects.filter(q1)

I would have expected qs1 and qs2 to yield the same result, but they
do not. They should both return the single item, but qs1 returns
nothing. The SQL for qs1 looks like:

SELECT "app_item"."id", "app_item"."title" FROM "app_item"
INNER JOIN "app_property" ON ("app_item"."id" =
"app_property"."item_id")
LEFT OUTER JOIN "app_property" T4 ON ("app_item"."id" = T4."item_id")
LEFT OUTER JOIN "app_propertyvalue" T5 ON ("app_property"."value_id" =
T5."id")
WHERE (("app_property"."value_id" = 1  AND "app_property"."key" =
'a' ) AND (T5."id" IS NULL AND T4."key" = 'b' ))

The first app_property join corresponds to q1, and the second
corresponds to q2. However, the app_propertyvalue join (corresponding
to the isnull from q2) refers to app_property.value_id (i.e. q1)
instead of T4.value_id (i.e. q2). I think fixing #10790 would fix
this, since removing the join to app_propertyvalue and simply checking
"T4.value_id IS NULL" works as expected, but I'm not sure if this is a
separate problem worthy of another ticket or not. Also (less
importantly), since both criteria (q1 and q2) are checking
props__key='something', I would imagine both joins could be INNER
JOINs, but the isnull seems to promote the second to a LEFT OUTER
JOIN.

I guess I was just wondering if I should open a separate ticket for
this, or is this simply a symptom of #10790? Or am I missing
something?

Regards,
Dan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Adding signing (and signed cookies) to Django core

[Dan Watson]

On Sep 24, 3:46 pm, Simon Willison  wrote:

> > 1) request.unsign_cookie('foo') -- This breaks the parallelism with
> > existing cookies.  Sometimes we'll be doing request.COOKIES['foo'] and
> > sometimes we'll be doing request.unsign_cookie('foo').
>
> If we were going to do that, it would make sense to NOT have set_cookie
> (... sign=True) as the API for setting one. We could achieve
> parallelism with something like this:
>
> response.sign_cookie('key', 'value')
> ...
> value = request.unsign_cookie('key')
>
> You can still read request.COOKIES directly, but you'll get the raw,
> signed value. That API doesn't look too ugly to me.

What about having something like:

request.get_cookie(key, signed=True)

that would fall back to returning request.COOKIES[key] when
signed=False? It creates a second way to access request cookies, but
introduces parallelism with the response.set_cookie method.

Dan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #3182 -- model instance update() method and QuerySet update_or_create() method

[Dan Watson]

On Mar 15, 4:40 pm, Jacob Kaplan-Moss 
wrote:
> We take a slightly more reasonable approach to backwards compatibly as
> detailed 
> athttp://docs.djangoproject.com/en/dev/misc/api-stability/#what-stable-
> Read that carefully: nothing there prevents us from adding a method
> named update.

Gotcha. I'll have to pursue http://code.djangoproject.com/ticket/5741
again :-)

Dan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #3182 -- model instance update() method and QuerySet update_or_create() method

[Dan Watson]

On Mar 15, 1:12 pm, "Gary Wilson Jr."  wrote:
> What do you think?

Wouldn't this be a backwards-incompatible change at this point? It
would clash with any model fields named "update".

Dan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: uploaded file permissions

[Dan Watson]

On Aug 20, 3:34 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Wed, 2008-08-20 at 12:13 -0700, Dan Watson wrote:
> > On Aug 14, 5:29 pm, Dan Watson <[EMAIL PROTECTED]> wrote:
> > > As mentioned a few times in #2070, uploaded files large enough to be
> > > streamed to a temporary file get created with a mode of 0600, as per
> > > python's tempfile.mkstemp. This causes two problems:
>
> > > 1. Files uploaded into memory and saved to disk respect the umask, so
> > > uploads could have different permissions based on how big they are.
>
> > > 2. If the webserver user and django user do not match (such as when
> > > running an external FastCGI process), the webserver can no longer
> > > serve uploaded files.
>
> > > I got around this issue by subclassing FileSystemStorage._save to call
> > > os.chmod, but it seems there should either be a setting (eg:
> > > FILE_UPLOAD_MODE), or a mode that respects the current umask should be
> > > specified when saving (and moving) the files. I can create a ticket
> > > and patch either way.
>
> > This inconsistency seems like a bug to me, so I've created ticket
> > #8454 [1] with a patch that implements a FILE_UPLOAD_PERMISSIONS
> > setting, and marked it as 1.0. This seemed like the least intrusive
> > way to go, and the default behavior is unchanged.
>
> Why did you go with another setting rather than using the umask (which
> hopefully would be set correctly -- the fastcgi server handles that
> now)? Is there some technical difficulty I'm not seeing here that means
> we can't get the umask and use that? It would seem nicer than having to
> worry about another setting that can get out of sync with things if it
> was possible.

I guess I was envisioning a situation where you'd want to chmod
uploaded files differently than other sorts of files your application
may be writing out. Also, it appears the only way to get the current
umask is by setting it at the same time, which seems like a good way
to introduce a race condition (set to X to retrieve Y, another caller
gets X instead of Y).

> I'm not disputing (yet :-)) that your approach is correct. Rather,
> wondering what I'm missing.

I don't think either solution is bad, so long as there is a solution
and it's documented. I'm not heavily in favor of one or the other, so
if you'd rather see the current umask used, I'll submit a new patch.
Another option would be to give the setting a reasonable default value
(likely 0600, 0644, or 0640).

Regards,
Dan
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: uploaded file permissions

[Dan Watson]

On Aug 14, 5:29 pm, Dan Watson <[EMAIL PROTECTED]> wrote:
> As mentioned a few times in #2070, uploaded files large enough to be
> streamed to a temporary file get created with a mode of 0600, as per
> python's tempfile.mkstemp. This causes two problems:
>
> 1. Files uploaded into memory and saved to disk respect the umask, so
> uploads could have different permissions based on how big they are.
>
> 2. If the webserver user and django user do not match (such as when
> running an external FastCGI process), the webserver can no longer
> serve uploaded files.
>
> I got around this issue by subclassing FileSystemStorage._save to call
> os.chmod, but it seems there should either be a setting (eg:
> FILE_UPLOAD_MODE), or a mode that respects the current umask should be
> specified when saving (and moving) the files. I can create a ticket
> and patch either way.

This inconsistency seems like a bug to me, so I've created ticket
#8454 [1] with a patch that implements a FILE_UPLOAD_PERMISSIONS
setting, and marked it as 1.0. This seemed like the least intrusive
way to go, and the default behavior is unchanged.

Regards,
Dan

[1] http://code.djangoproject.com/ticket/8454
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

database schema support

[Dan Watson]

I know this has been touched on a couple times, but I have an itch for
database schema support I'd like to scratch. The relevant ticket it
#6148 [1]. I was thinking the Options class would grow a db_schema
field, which would default to settings.DEFAULT_DATABASE_SCHEMA. Then
everywhere that currently deals with db_table will also take db_schema
into account, if specified.

My initial thought was that it'd be relatively easy to introduce a new
Options property, db_quoted_name (or something), that would return an
already-quoted "schema"."table" string, then use this in place of
db_table pretty much everywhere. This will avoid having to muck with
the join tuples and aliases in django.db.models.sql.query, since
db_quoted_name will be canonical (db_table no longer will be). There
will be certain places (index and sequence SQL generation come to
mind) where the schema will need to be dealt with separately. Also,
the table introspection code will need to return schema names as well
as table names, if applicable.

The ticket mentions app-level schema support, which would be nice, but
I think can be punted until #3591 [2] lands. Then the app class could
take a schema argument that would be propagated to the models when the
app is loaded, for models that do not already define db_schema.

Any thoughts or suggestions?

[1] http://code.djangoproject.com/ticket/6148
[2] http://code.djangoproject.com/ticket/3591
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Subversion Access

[Dan Watson]

Questions like this should be directed to the django-users mailing
list.

That said, this is probably caused by your proxy server not
understanding how to handle subversion's requests. See
http://subversion.tigris.org/faq.html#proxy or search the mailing
lists - this has come up before.

On May 7, 5:00 pm, Rahein <[EMAIL PROTECTED]> wrote:
> I am trying to get the Django Trunk downloaded and this is my first
> excursion into using Subversion.
>
> I followed the directions in the installing guide, but when I run the
> command in subversion's bin folder:
>
> svn cohttp://code.djangoproject.com/svn/django/trunk/django-trunk
>
> I get these error message:
>
> svn: REPORT request failed on '/svn/!svn/vcc/default'
> svn: REPORT of '/svn/!svn/vcc/default': 400 Bad Request (http://
> code.djangoproject.com)
>
> Thanks for your help again.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: QuerySet.exists()

[Dan Watson]

On Oct 17, 6:47 pm, "James Bennett" <[EMAIL PROTECTED]> wrote:

> You want to read this thread (first result when searching the group
> archive for "queryset exists"):
>
> http://groups.google.com/group/django-developers/browse_frm/thread/54...

I mentioned that thread at the bottom of my post, because they are
unrelated. That thread discuses .exists as a method of determining if
a QuerySet contains at least one row.

However, I have no idea how I missed the fact that you could query
backwards across relationships. I think a method for specifying EXISTS
clauses may still be useful in some cases, but I'll keep that in my
pocket for a rainy day.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

QuerySet.exists()

[Dan Watson]

I was wondering what people thought of adding a .exists() method to
querysets that would take another QuerySet, and add a "WHERE EXISTS
(SELECT ...)" to the original query. It would require that the passed-
in QuerySet be joinable with the original QuerySet via a ForeignKey
relationship. Take the following example:

class Record (models.Model):
name = models.CharField( max_length=100 )
active = models.BooleanField()

class RecordValue (models.Model):
record = models.ForeignKey( Record, related_name='values' )
value = models.TextField()

Now suppose you want to build a search engine. Searching for
Record.name='Dan' would be easy, and searching for Records with a
RecordValue.value='django' is also [slightly less] easy. But the
latter requires using a RecordValue queryset, and extracting the
records from it. So I thought it would be nice to be able to do
something like:

Record.objects.filter( name='Dan' ).exists( RecordValue.objects.filter( 
value='django' ) )

I'm not sure how difficult it would be to implement, but I'd be
willing to take a stab at it (against qs-rf once it's merged and/or
stabilized) if it's of interest to anyone.

Dan

Note: This is unrelated to
http://groups.google.com/group/django-developers/browse_frm/thread/5490f8736cc54c27/893ba34ec554bfc3
(although, interestingly, the name "any" was suggested in the that
thread, which is what SQLAlchemy calls what I describe above - don't
really care what the method is called)


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: proposal: update QuerySet.get() to return a default if object does not exist?

[Dan Watson]

Just to ping this again (since I think it's super-convenient and don't
want it to get lost), I've created ticket 5741 with a patch and doc
changes. The patch would look the same for the queryset-refactor
branch as it stands now, too.

http://code.djangoproject.com/ticket/5741

Dan

On Aug 23, 2:09 am, "Adrian Holovaty" <[EMAIL PROTECTED]> wrote:
> On 8/22/07, Russell Keith-Magee <[EMAIL PROTECTED]> wrote:
>
> > So - I would suggest holding off for a week or two (to let the dust
> > settle on the refactor), and then poke this issue again. For the
> > record, I think the general idea has merit, we just need to finess
> > some details.
>
> Yeah, we should wait until the query.py refactoring is done, but I'm
> +1 on this change. It's a pain to have to wrap things in try/excepts
> all of the time.
>
> Of the options Rob pointed out, I prefer the last -- giving the get()
> method an optional "default" parameter, instead of creating new
> methods like get_or_default() or get_or_none(). We can easily get
> around the fact that it clashes with fields named "default" -- if you
> have a field named "default" and want to do an exact lookup, just use
> "default__exact" instead of "default". That seems like a reasonable
> solution.
>
> This would be backwards-incompatible for anybody who currently has a
> field named "default" and does an exact lookup. But I think that's OK,
> as long as we document it and get the word out.
>
> Adrian
>
> --
> Adrian Holovaty
> holovaty.com | djangoproject.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: None != Null?

[Dan Watson]

Just thought I'd mention one other point to consider: at least in
postgresql, there is a configuration option to transform "= NULL" to
"IS NULL" on the fly. [1] So if you're using "= NULL" (which is an
error in virtually all cases) you may get different results on
different databases anyway. Using "IS NULL" is unambiguous.

[1] 
http://www.postgresql.org/docs/8.2/static/runtime-config-compatible.html#GUC-TRANSFORM-NULL-EQUALS

On Jul 17, 11:24 am, David Cramer <[EMAIL PROTECTED]> wrote:
> I'm still not quite sure what the point of having None not mean NULL
> is. '' is not None, False is not None. If this is what it's currently
> doing then the ORM handles it incorrectly. Yes, it may be a backwards
> incompatible change, but who relies on None? I've yet to see
> myfield=None pull any results from my database, whether the field was
> an empty string or a NULL value.
>
> I'd also like to point out the obvious that Django does make quite a
> few backwards incompatible changes, and this one, I don't think has
> any serious chance to cause a ruckus with current code. It's a hell of
> a lot easier than changing auto_now to datetime.datetime.now
> everywhere ;)
>
> If Django's ORM can't evolve then Django can't evolve.
>
> On Jul 17, 7:54 am, "Marty Alchin" <[EMAIL PROTECTED]> wrote:
>
> > On 7/17/07, Michael Radziej <[EMAIL PROTECTED]> wrote:
>
> > > b) the new meaning will lead newbies to a wrong understanding of SQL, 
> > > since they will
> > > assume that 'WHERE col=NULL' means 'WHERE col is NULL', which it does
> > > not.
>
> > I've been avoiding this topic for the most part, but I'd like to put
> > my two cents in on this comment. To my knowledge, much of the point of
> > Django is to avoid having to deal with SQL wherever possible, so I
> > find it strange to think that Django should try to be a tool for
> > teaching SQL to the uninitiated. The ORM is a way to map logical
> > Python code to function SQL statements, that's all. If it makes sense
> > in Python and works properly in SQL, it's a success.
>
> > If users want to learn SQL, they shouldn't be using Django to do it.
> > And if they do, that's on them if they run into problems. At least,
> > that's the way it seems to me. Ignore it if you like.
>
> > -Gul


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Google Sitemaps

[Dan Watson]

[EMAIL PROTECTED] wrote:
> I've installed it and it's very nice. I've moved "sitemap" to contrib -
> is it ok? :) because there is one minor problem is with sitemap.xml -
> it try to load it from:
> 
> Django tried loading these templates, in this order:
> Using loader django.template.loaders.filesystem.load_template_source:
> templates/sitemap.xml (File does not exist)
> Using loader
> django.template.loaders.app_directories.load_template_source:
> /usr/lib/python2.4/site-packages/Django-0.95-py2.4.egg/django/contrib/admin/templates/sitemap.xml
> (File does not exist)
> 
> when i put sitemap.xml in my templates dir (templates/) it works nicely
> :)

I guess it's probably worth mentioning you need to put
django.contrib.sitemap in your INSTALLED_APPS, and you need
django.template.loaders.app_directories.load_template_source in your
TEMPLATE_LOADERS for it to work out of the box. Or you can just do what
you did, and copy the templates and skip the other two steps :-)

> PS. how can i combine two querysets and set different priority for
> them? :)

items() just needs to return a list, so you could just build a list by
evaluating both querysets and concatenating them. Then in priority(),
you could check to see the type of the passed-in object and return a
priority accordingly.

A cleaner approach, however, would just to just create two Sitemap
subclasses, each with a "priority" class attribute. I suppose I could
also update GenericSitemap to take a default_priority (and a default_*
for that matter)... what do people think about that?

Dan


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Google Sitemaps

[Dan Watson]

Deryck Hodge wrote:
> Hi, Dan.
>
> On 8/30/06, Dan Watson <[EMAIL PROTECTED]> wrote:
> > Not sure if people will use it, but suggestions and feedback are always
> > welcome!
>
> I had to do a sitemap generator for a client from work recently -- my
> code is pre-mr due to older version of Django we're on.  One of the
> problems I had was that I needed to specify priority on a fairly
> granular level, not to mention quite a large number of URLs.  I only
> glanced at your code, so my appologies if your docs cover this, but do
> you have any means for configuring priority per URL?  For example, say
> one blog entry in your blog sitemap is important to you, but unindexed
> or misrepresented in Google's index, and you want to specify a 0.8
> priority for it, but 0.5 for other blog urls.

Sure, this works just like syndication (or the example in my docs for
the lastmod method). items() returns a list of objects (blog entries in
your case), and if a priority() method exists in your Sitemap, it is
called with each object. Chances are, your BlogSitemap class would look
something like this:

class BlogSitemap (Sitemap):
changefreq = "monthly"
def items( self ):
return Entry.objects.all()
def lastmod( self, obj ):
return obj.pub_date
def priority( self, obj ):
return obj.priority

Dan

> Porbably not something that affects a lot of folks, but just something
> to consider from a real world use case.
> Cheers,
> deryck
>
> --
> Deryck Hodgehttp://www.devurandom.org/
> Web Developer, Naples News http://www.naplesnews.com/
> Samba Team http://www.samba.org/


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Google Sitemaps

[Dan Watson]

Jacob Kaplan-Moss wrote:
> On Aug 30, 2006, at 11:51 AM, Dan Watson wrote:
> > Just wanted to draw some attention to
> > http://code.djangoproject.com/ticket/2628 - it's a contrib app I've
> > written to automatically generate sitemap.xml files for Google. I've
> > also written about it here:
> > http://theidioteque.net/blog/2006/08/29/google-sitemaps/
>
> Great stuff, Dan!
>
> My only quibble is the recommendation in the docs to call
> ``ping_google()`` from the ``save()`` method.  I think the idea of
> setting off a HTTP request from within a ``save()`` method isn't such
> a grand idea; network latency could cause the save to take too long,
> or the request could raise an exception causing the object not to get
> saved...  I'm not sure about what would be better, but it does rub me
> the wrong way.

Yeah, I wasn't too concerned about the latency, since a save operation
taking a little longer isn't so bad compared to a read operation. My
concern would be more along the lines of sending a lot of pings
unnecessarily. A smarter approach for most sites would be to call it
daily (or at some interval) from a cron job.

I'm fine with with taking that piece out of the documentation, though.

Dan


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Google Sitemaps

[Dan Watson]

Just wanted to draw some attention to
http://code.djangoproject.com/ticket/2628 - it's a contrib app I've
written to automatically generate sitemap.xml files for Google. I've
also written about it here:
http://theidioteque.net/blog/2006/08/29/google-sitemaps/

Not sure if people will use it, but suggestions and feedback are always
welcome!

Dan


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Validation Aware Models and django.forms on steroids

[Dan Watson]

> I'm afraid that I find the idea in Brantley Harris's proposal of
> raising a Form as an exception as a form of flow control really
> counter-intuitive. Apologies and it's just my opinion of course, but it
> has a sort of  "too clever" feel to me. Also, exceptions are very
> expensive, and I don't see the practical benefits of that usage pattern
> over something like:

Actually that was one of my favorite pieces. I think it captures what's
going on in a very intuitive way: try to create/update, if that fails,
redisplay with errors. Maybe a good solution to the "too clever"
feeling would be to, instead of raising a Form as an exception, raising
a real error, with a property/accessor to get at a partially-completed
form?

try:
  m = Poll.CreateManipulator()
  poll = m.save( req.POST ) #, req.user?
  return HttpResponseRedirect( '/poll/%d/' % poll.id )
except ManipulatorError, e:
  # print e.errors
  return render_to_response( 'poll/create.html', {'form': e.form} )


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: [Fw]The Python Web Framework

[Dan Watson]

> In my experience such simple behavior is rarely needed. When you
> actually need a pool it means that your app become pretty large so it
> requires not only static pool but also other settings like minimum spare
> connections, maximum spare connections, keep-alive timeouts etc... So
> this becomes a complex matter in itself and I'd rather Django not to
> reinvent this.

What about cases where running an external connection pooler is not an
option, such as shared hosting environments?

> For me this is very similar to the recommendation to use memcached as a
> cache backend instead of mimicking its rich functionality inside Django.

But django still implements file/database/memory caching that does not
require memcached. Recommending a particular setup as "best practice"
is fine and good, but there should be alternatives for those who can't
use or don't need the "recommended" setup.

Ideally, it seems django should offer simple connection pooling with 2
options: number of connections and an on/off switch. That would satisfy
the needs of some/most, and for those that need something more robust,
look into an external pooler.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Multi-db branch feedback/questions

[Dan Watson]

Another use case to mull around:

I'm currently using django for basically everything *but* it's ORM, in
large part due to not supporting multiple databases. We have one
software package with many nearly-identical databases (for different
customers). The models would be the same, but I'd want to decide where
to save them at runtime, not as a Meta option. Creating a custom
manager for each database would work, but with 10+ databases, it could
pollute the namespace, and be tedious to write.

> > I posted a couple of example use-cases in response to Malcolm.
> > Manually setting ``db_connection`` is ugly but workable; I'll think
> > some more about a more elegant way of doing it.

What about making Manager callable? It could take a database
identifier, or even an object with connection info, and return a new
Manager instance with the specified database info. This would let you
keep using Model.objects with a default database, or for the case where
multiple databases are not specified. But then you could do something
like this:

Person.objects(db).filter(...)

Dan


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Moving auth and core models to contrib -- and dependencies

[Dan Watson]

I'm definitely +1 on splitting out the core and auth models/tables. If
installing them is just as simple as running init and installing the
admin app is now, what's the big deal? For those who want it, it's no
harder, and you'd be able to install the admin without even realizing
you're installing the core and auth models. But for those who don't
want it, you're stuck removing them by hand.

Also, having tables which aren't being used may not be a big deal to
your average user, but in a situation where you want to implement a
django application over an already existing database, especially in a
corporate environment, having pointless tables may not be an option.