Re: Custom filter() operators

[Jay Deiman]

Malcolm Tredinnick wrote:
> On Fri, 2009-03-06 at 08:00 +0900, Russell Keith-Magee wrote:
> [...]
>> However, I agree that this would be a useful feature to add (or
>> improve). This isn't something we want to be trivial to accomplish,
>> but it should be a lot easier for people who are adding fields that
>> have unusual operator requirements. I haven't dug into the details to
>> work out exactly how we would approach this - any proposals are
>> welcome, especially if you can work through the details to see what
>> might be possible.
> 
> It's certainly on my long-term list (but no closer than that at the
> moment). The hard part is keeping the Field classes and the database
> storage layer reasonably separate. If people want to combine the two in
> their own custom Field classes, that's fine, but I don't want to
> introduce even tighter coupling than we already have in Django (there's
> some leakage of SQL-awareness into the Field layer, but it's minimal),
> because the idea is to make it easy to replace the SQL storage with
> something else. So it's the internal API that is the largely non-trivial
> part.
> 
> Right now, the easiest way to do this stuff is to forget about doing it
> as a custom lookup type and do it as a custom Q-like object. Then you
> could write something like
> 
> Foo.objects.filter(CIDR('my_field', '1.2.3.4'))
> 
> Here, CIDR is something like a Q class. Have a look at
> django.db.models.sql.Query.add_q() for how anything with add_to_query()
> and as_sql() methods are handled.

Well, as it stands now, I just patched the Django 1.0.2-final source to 
add the necessary lookups, which is below for any who are interested. 
Keep in mind this was a simple patch to just add the functionality I 
need for my current project, not necessarily "top notch" work.

My long term plan, when I finish my current project is to submit this 
patch, plus my InetField and CidrField (and their related form field 
subclasses), to the project.  I plan to make this backend independent by 
defaulting to a CharField for any backend, other than Postgres, on these 
two fields.  This would at least make these (very useful) types 
available to all.  Since I will be using these quite extensively within 
my current project, I will be able to do code tweaking and testing, as 
well as some actual documentation, before submitting these to the Django 
upstream for inclusion.

As noted previously, I think there would be a great benefit to allowing 
the easy addition of custom db filters and I would be happy to help out 
with this after my current project I'm working on.

Thanks for the help and suggestions,

Jay

-- 
Jay Deiman

\033:wq!

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Custom filter() operators

[Malcolm Tredinnick]

On Fri, 2009-03-06 at 08:00 +0900, Russell Keith-Magee wrote:
[...]
> However, I agree that this would be a useful feature to add (or
> improve). This isn't something we want to be trivial to accomplish,
> but it should be a lot easier for people who are adding fields that
> have unusual operator requirements. I haven't dug into the details to
> work out exactly how we would approach this - any proposals are
> welcome, especially if you can work through the details to see what
> might be possible.

It's certainly on my long-term list (but no closer than that at the
moment). The hard part is keeping the Field classes and the database
storage layer reasonably separate. If people want to combine the two in
their own custom Field classes, that's fine, but I don't want to
introduce even tighter coupling than we already have in Django (there's
some leakage of SQL-awareness into the Field layer, but it's minimal),
because the idea is to make it easy to replace the SQL storage with
something else. So it's the internal API that is the largely non-trivial
part.

Right now, the easiest way to do this stuff is to forget about doing it
as a custom lookup type and do it as a custom Q-like object. Then you
could write something like

Foo.objects.filter(CIDR('my_field', '1.2.3.4'))

Here, CIDR is something like a Q class. Have a look at
django.db.models.sql.Query.add_q() for how anything with add_to_query()
and as_sql() methods are handled.

Regards,
Malcolm




--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Custom filter() operators

[Russell Keith-Magee]

On Fri, Mar 6, 2009 at 5:51 AM, Jay Deiman  wrote:
>
> I'm currently trying to create a custom Field subclass for the Postgres
> specific "inet" and "cidr" types.  I have been able to create the
> classes and use them without issue so far.  The problem I am running
> into is that I would like to add some custom query filters for use with
> the filter() method.

This is one of those things that falls in the "possible, but not
particularly easy" category.

GeoDjango, for example, adds a whole lot of new filter types to handle
spatial fields:

http://geodjango.org/docs/db-api.html#spatial-lookup-types

GeoDjango does this is by subclassing the internal SQL Query class,
and overriding the way that the filter functions are evaluated. There
aren't any simple step-by-step instructions for doing this - you'll
need to pick apart the code a bit to see how all the parts fit
together.

However, I agree that this would be a useful feature to add (or
improve). This isn't something we want to be trivial to accomplish,
but it should be a lot easier for people who are adding fields that
have unusual operator requirements. I haven't dug into the details to
work out exactly how we would approach this - any proposals are
welcome, especially if you can work through the details to see what
might be possible.

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Custom filter() operators

[Jay Deiman]

I'm currently trying to create a custom Field subclass for the Postgres 
specific "inet" and "cidr" types.  I have been able to create the 
classes and use them without issue so far.  The problem I am running 
into is that I would like to add some custom query filters for use with 
the filter() method.

What I'm trying to do is make usable 4 filter options specific to these 
Fields.  Here are the names I'd like to use and their postgres operators:

'cidrct': '>> %s'
'cidrcte': '>>= %s'
'cidrctw': '<< %s'
'cidrctwe': '<<= %s'

I was originally hoping to somehow extend the "operators" dictionary 
listed in:

django.db.backends.postgresql_psycopg2.base.DatabaseWrapper

What exactly do I need to do to add those 4 filter opts so that if I had 
the following Model, I could use something to the effect of 
Netblock.objects.filter(inet__cidrct="1.2.3.4") and get back a Netblock 
entry of "1.2.3.0/24", for example:

class Netblock(BaseModel):
 inet = InetField()

Thanks,

Jay

-- 
Jay Deiman

\033:wq!

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---