Re: Composite Primary Keys
On Tue, Feb 16, 2016 at 05:38:12PM -0800, Cristiano Coelho wrote: > Hello there, > > What's the status for this? This > (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 years > old (last edit) and the links on it are even older. Googling around only > gave me some very old projects so it wasn't good neither. Hi Cristiano, I personally have spent in total perhaps about a year (or even more) working on this (which includes two Summer of Code programs). Unfortunately, my work was spread over more than three years, which means a significant chunk of effort went into keeping up with changes in Django and, ultimately, that's the part at which I failed. Twice. (With a patch of this magnitude, all you need is to fall behind by three weeks, and it gets very difficult and/or time-consuming to merge new changes into it.) The latest effort was launched by Thomas Stephenson, and resulted in the following DEP: https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst, There was even a pull request, but eventually, Thomas withdrew his patch. You can find the corresponding thread on this mailing list here: https://groups.google.com/d/msgid/django-developers/ed326cce-6784-429b-869b-f6f66d3c77fd%40googlegroups.com I wasn't around at that particular time, and I haven't yet found the time to carefully read that thread and let it sink in, but I can give a summary of my observations from 2013, when I last worked on this. If you're interested, you can take a look at the wall of text I wrote as my GSoC proposal in 2013: https://gist.github.com/konk/5408673. You might also want to search this mailing list for previous threads about this topic. I seem to recall that just the most basic functionality of CompositeField was the easy part. At that time (before migrations landed), it meant mostly just making some parts of Django play nicer with virtual fields (which was an almost entirely unexplored concept in Django back then), some hackery for query lookups (since we didn't have the nice API for custom lookups and transforms), and a fancy class to represent composite values in model instances. Whipping up something like this takes a few days, at most weeks, of focused work, and you can have something sort of usable, with many sharp edges that you have to keep in mind. Things start to get ugly when you try to make CompositeField play nice with the rest of the ORM, and the most obvious troublemaker is ForeignKey. Composite ForeignKey is something that requires a big refactor of the way related fields work, but even with a simple ForeignKey to a unique column on a model with a composite primary key, you'd start stumbling on errors with Python-level delete cascades, prefetching and such -- those require an overhaul of how __in lookups are handled. Then there are things like generic foreign keys, database migrations, introspection, and potentially other features that would need to be taken into account. Most of them could be postponed to a later time, but some of them would most likely have to be part of the initial patch. Anyway, with the recent _meta refactor, reimplementing ForeignKey as a virtual field should hopefully be easier now, although no one will know for sure unless someone actually steps up and tries to do it. I certainly hope that I'll be able to tackle this once again, but I cannot make any promises. Cheers, Michal -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20160222102153.GH880%40konk.org. For more options, visit https://groups.google.com/d/optout. signature.asc Description: Digital signature
Re: Composite Primary Keys
Sounds cool, I think we would totally use that. On Wednesday, February 17, 2016 at 1:55:15 AM UTC-8, Roger Gammans wrote: > > Hi, > > > We've got some patches that we are using in production, against > 1.8 . Out long term aim would be to get them ready for a PR , but I they > are missing all sorts of things, such unit tests and migrations (we > don't use migrations at the moment - long story, so these haven't been > tested). The other thing which I think is maybe is some of the > prefetch_related API, I've had difficulties with it and just worked > around it in my app. > > And it really needs another user at least to kick the tires which we > aren't exercising - we mainly use because we have a lot of intersection > tables of natural / 'foreign' keys and a legacy model which didn't add a > serialized Pk for these. > > I'd also like to know how interested the core developers would be in > merging some support for this. > > > > On Tue, 2016-02-16 at 17:38 -0800, Cristiano Coelho wrote: > > Hello there, > > > > What's the status for this? This > > (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 > > years old (last edit) and the links on it are even older. Googling > > around only gave me some very old projects so it wasn't good neither. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Django developers (Contributions to Django itself)" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to django-develop...@googlegroups.com . > > To post to this group, send email to > > django-d...@googlegroups.com . > > Visit this group at https://groups.google.com/group/django-developers. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/django-developers/b2873ece-39bb-4536-b23d-d988c7122204%40googlegroups.com. > > > > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/098addaa-a397-4016-913c-e1a3037d34e1%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Composite Primary Keys
Hi, We've got some patches that we are using in production, against 1.8 . Out long term aim would be to get them ready for a PR , but I they are missing all sorts of things, such unit tests and migrations (we don't use migrations at the moment - long story, so these haven't been tested). The other thing which I think is maybe is some of the prefetch_related API, I've had difficulties with it and just worked around it in my app. And it really needs another user at least to kick the tires which we aren't exercising - we mainly use because we have a lot of intersection tables of natural / 'foreign' keys and a legacy model which didn't add a serialized Pk for these. I'd also like to know how interested the core developers would be in merging some support for this. On Tue, 2016-02-16 at 17:38 -0800, Cristiano Coelho wrote: > Hello there, > > What's the status for this? This > (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 > years old (last edit) and the links on it are even older. Googling > around only gave me some very old projects so it wasn't good neither. > > -- > You received this message because you are subscribed to the Google > Groups "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to > django-developers@googlegroups.com. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/b2873ece-39bb-4536-b23d-d988c7122204%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1455702835.27233.10.camel%40gammascience.co.uk. For more options, visit https://groups.google.com/d/optout.
Composite Primary Keys
Hello there, What's the status for this? This (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 years old (last edit) and the links on it are even older. Googling around only gave me some very old projects so it wasn't good neither. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b2873ece-39bb-4536-b23d-d988c7122204%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Composite primary keys
On Thu, Mar 31, 2011 at 09:54:49AM -0700, Christophe Pettus wrote: > What concerns me about composite fields is that they seem to be a > lot of Python machinery just to accomplish the goal of allowing this > annotation. If they were super-useful in their own right, that > would be one thing, but I'm not sure that I see the utility of them > absent indexes and foreign keys. I'm also bothered, perhaps > excessively, about having two different ways of getting at the same > field in the model just to support this. Just an observation, there already are two ways of getting at the same field in case of primary keys... > So, another proposal: > > In the foreign key case, just extending the ForeignKey syntax to > allow for multiple related fields makes the most sense: > > overThere = models.ForeignKey(OtherModel, to_field=('first_name', > 'last_name', )) > > For indexes on the table for the model, include the declaration in > the Meta class, since that's the obvious place to stick indexing: > > class SomeModel: > > class Meta: > primary_key = 'some_field' > indexes = ['some_field', 'some_other_field', ('field1', > '-field2', ), ] > raw_indexes = [ 'some_invariant_function(some_field)' ] > > (This was proposed by someone else, and isn't original to me; > apologies that I can't find the email to give credit.) > > Of course, the existing syntax would still work as a shortcut for > primary_key and indexes. > > Thoughts? One thing I'm missing in this proposal is the behavior of the pk property. Since the primary key is a tuple, I can't imagine any other representation in .pk than some flavor of a tuple. And to make things like SomeModel.objects.get(pk=some_value) possible, we'd still have to implement most of the functionality of a composite field. Michal Petrucha signature.asc Description: Digital signature
Re: Composite primary keys
On Mar 21, 2011, at 12:20 PM, Jacob Kaplan-Moss wrote: > I think we're talking slightly different concerns here: I'm mostly > interested in the Python-side API, and to my eyes a composite field > matches more closely what's happening on the Python side of things. I agree 100%! I think I'm just drawing a different conclusion from that point, which is that indexes are more metadata on the database rather than a critical part of the Python API: In an imaginary perfect database (like, say, the SQL spec envisions), we wouldn't need to talk about indexes as all. The more I think about it, the less I like including this directly in the field declaration part of the model, including my Index type proposal. It just doesn't seem to belong there. What concerns me about composite fields is that they seem to be a lot of Python machinery just to accomplish the goal of allowing this annotation. If they were super-useful in their own right, that would be one thing, but I'm not sure that I see the utility of them absent indexes and foreign keys. I'm also bothered, perhaps excessively, about having two different ways of getting at the same field in the model just to support this. So, another proposal: In the foreign key case, just extending the ForeignKey syntax to allow for multiple related fields makes the most sense: overThere = models.ForeignKey(OtherModel, to_field=('first_name', 'last_name', )) For indexes on the table for the model, include the declaration in the Meta class, since that's the obvious place to stick indexing: class SomeModel: class Meta: primary_key = 'some_field' indexes = ['some_field', 'some_other_field', ('field1', '-field2', ), ] raw_indexes = [ 'some_invariant_function(some_field)' ] (This was proposed by someone else, and isn't original to me; apologies that I can't find the email to give credit.) Of course, the existing syntax would still work as a shortcut for primary_key and indexes. Thoughts? -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mon, Mar 21, 2011 at 02:20:41PM -0500, Jacob Kaplan-Moss wrote: > On Mon, Mar 21, 2011 at 2:33 AM, Christophe Pettus wrote: > > I'd like to make one more pitch for a slightly different > > implementation here. My concern with CompositeField isn't based > > on the fact that it doesn't map one-to-one with a field in the > > table; it's that it doesn't have any of the semantics that are > > associated with a field. In particular, it can't be: > > > > - Assigned to. > > - Iterated over. > > - Or even have a value. > > Obviously there's no code here yet, so we don't know exactly. I'd also > be -1 on an implementation of a CompositeField that didn't have those > values. However, it's reasonably easy to come up with a CompositeField > that is assignable, iterable, and has values. Here's the basics:: > [snippet] > > Fields even sorta get saved correctly to the DB with just these few > lines of code. Of course there's a lot missing here to correctly > handle actual composite keys -- this tiny example won't work in > querysets, for example -- but the basics of the Python-side behavior's > right there. This is exactly where I would start. One question though: should I use namedtuple in here or should we try to keep compatibility with python < 2.6? Maybe a module like django.utils.namedtuplecompat providing a fallback implementation? > > My suggestion is to create an Index type that can be included in a > > class just like a field can. > > I think we're talking slightly different concerns here: I'm mostly > interested in the Python-side API, and to my eyes a composite field > matches more closely what's happening on the Python side of things. > Python's not generating an index, after all, so using something called > "index" for compositing multiple attributes together seems weird to > me. But at the DB level, "index" makes perfect sense. Thing is, we've > always tried to make Django's APIs behave well in Python *first*, and > then think about the DB concerns. And again, to me "composite field" > matches more closely the behavior we want out of the Python side of > things. > > All that said, there's a lot to like about your Index proposal. > Perhaps there's a way we can merge these two things together somehow? The composite index could be achieved by the standard field option Field.db_index applied to a CompositeField. This would still be consistent with the rest of the API since it would not require any new construct to do this thing. To sum up, I believe the CompositeField could be a general solution to both composite primary keys and composite indexes. Michal Petrucha signature.asc Description: Digital signature
Re: Composite primary keys
On Mon, Mar 21, 2011 at 2:33 AM, Christophe Pettus wrote: > I'd like to make one more pitch for a slightly different implementation here. > My concern with CompositeField isn't based on the fact that it doesn't map > one-to-one with a field in the table; it's that it doesn't have any of the > semantics that are associated with a field. In particular, it can't be: > > - Assigned to. > - Iterated over. > - Or even have a value. Obviously there's no code here yet, so we don't know exactly. I'd also be -1 on an implementation of a CompositeField that didn't have those values. However, it's reasonably easy to come up with a CompositeField that is assignable, iterable, and has values. Here's the basics:: class CompositeField(object): def __init__(self, *fields): self.fields = fields def contribute_to_class(self, cls, name): nt_name = "%s_%s" % (cls.__name__, name) nt_fields = " ".join(f.name for f in self.fields) self.nt = collections.namedtuple(nt_name, nt_fields) setattr(cls, name, self) def __get__(self, instance, owner): if instance: return self.nt._make(getattr(instance, f, None) for f in self.nt._fields) raise AttributeError("Composite fields only work on instances.") def __set__(self, instance, value): for (field, val) in zip(self.nt._fields, value): setattr(instance, field, val) It works, too:: class Person(models.Model): first_name = models.CharField(max_length=100) last_name = models.CharField(max_length=100) name = CompositeField(first_name, last_name) # ... >>> p = Person(first_name="Jacob", last_name="KM") >>> p.name : Person_name(first_name='Jacob', last_name='KM') >>> p.name.last_name : 'KM' >>> p.name = ("John", "Doe") >>> p.last_name : 'Doe' >>> for f in p.name: ..: print f ..: ..: John Doe Fields even sorta get saved correctly to the DB with just these few lines of code. Of course there's a lot missing here to correctly handle actual composite keys -- this tiny example won't work in querysets, for example -- but the basics of the Python-side behavior's right there. > My suggestion is to create an Index type that can be included in a class just > like a field can. I think we're talking slightly different concerns here: I'm mostly interested in the Python-side API, and to my eyes a composite field matches more closely what's happening on the Python side of things. Python's not generating an index, after all, so using something called "index" for compositing multiple attributes together seems weird to me. But at the DB level, "index" makes perfect sense. Thing is, we've always tried to make Django's APIs behave well in Python *first*, and then think about the DB concerns. And again, to me "composite field" matches more closely the behavior we want out of the Python side of things. All that said, there's a lot to like about your Index proposal. Perhaps there's a way we can merge these two things together somehow? Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 21, 1:20 pm, Michal Petrucha wrote: > > My suggestion is to create an Index type that can be included in a > > class just like a field can. The example we've been using would > > then look like: > > > class Foo(Model): > > x = models.FloatField() > > y = models.FloatField() > > a = models.ForeignKey(A) > > b = models.ForeignKey(B) > > > coords = models.CompositeIndex((x, y)) > > pair = models.CompositeIndex((a, b), primary_key=True) > > > We could have FieldIndex (the equivalent of the current > > db_index=True), CompositeIndex, and RawIndex, for things like > > expression indexes and other things that can be specified just as a > > raw SQL string. > > > I think this is a much better contract to offer in the API than one > > based on field which would have to throw exceptions left and right > > for most of the common field operations. > > I don't see how ForeignKeys would be possible this way. > In much the same way: class FooBar(Model): a = models.ForeignKey(A) b = models.ForeignKey(B) pair = models.ForeignKey(Foo, fields=(a, b)) Note that this is very close to what SQL does. If you have a composite unique index or composite foreign key you define the fields and then the index / foreign key. Though I don't know how much value that argument has in this discussion. You could add some DRY and allow a shortcut: class FooBar(Model): pair = models.ForeignKey(Foo) # a and b are created automatically. Now, to make things work consistently pair should be a field. But on the other hand when using a ModelForm, the pair should probably not be a field of that form. This is more clear in an example having a (city, state, country) primary key. These should clearly be separate fields in a form. In my opinion, if the composite structures are called fields or something else isn't that important. There are cases where composite structures behave like a field and some cases where they do not. The main problem is how the composite structures should behave in ModelForms and serialization, should they be assignable, how the relate to model __init__ method, should they be in model fields iterators, how they are used in QuerySets and so on. When these questions are answered it is probably easier to answer if the composite structures should be called fields or something else. - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mon, Mar 21, 2011 at 12:33:01AM -0700, Christophe Pettus wrote: > I'd like to make one more pitch for a slightly different > implementation here. My concern with CompositeField isn't based on > the fact that it doesn't map one-to-one with a field in the table; > it's that it doesn't have any of the semantics that are associated > with a field. In particular, it can't be: > > - Assigned to. > - Iterated over. > - Or even have a value. I disagree. The CompositeField would need to have a value to be able to implement a ForeignKey pointing to a modedel with a composite primary key. The CompositeField itself would be just a proxy to the actual atomic fields. You should be able to assign a tuple (or namedtuple) to it, specifying the actual values for the fields. Similarly, you'll be able to retrieve its value which is a tuple or a namedtuple. This way, the following code would work for composite primary keys the same way it works for simple keys: class CompositeModel(models.Model): a = models.IntegerField() b = models.IntegerField() key = models.CompositeField((a, b), primary_key=True) class ReferencingModel(models.Model): cm = models.ForeignKey(CompositeModel) cminstance = CompositeModel.objects.get(something) newref = ReferencingModel() newref.cm = cminstance.pk > My suggestion is to create an Index type that can be included in a > class just like a field can. The example we've been using would > then look like: > > class Foo(Model): >x = models.FloatField() >y = models.FloatField() >a = models.ForeignKey(A) >b = models.ForeignKey(B) > >coords = models.CompositeIndex((x, y)) >pair = models.CompositeIndex((a, b), primary_key=True) > > We could have FieldIndex (the equivalent of the current > db_index=True), CompositeIndex, and RawIndex, for things like > expression indexes and other things that can be specified just as a > raw SQL string. > > I think this is a much better contract to offer in the API than one > based on field which would have to throw exceptions left and right > for most of the common field operations. I don't see how ForeignKeys would be possible this way. Michal Petrucha signature.asc Description: Digital signature
Re: Composite primary keys
Am 21.03.2011 um 08:33 schrieb Christophe Pettus: > I'd like to make one more pitch for a slightly different implementation here. > My concern with CompositeField isn't based on the fact that it doesn't map > one-to-one with a field in the table; it's that it doesn't have any of the > semantics that are associated with a field. In particular, it can't be: > > - Assigned to. > - Iterated over. > - Or even have a value. You would be able to use composite fields normally (as in "normal django field"): >>> foo = Foo.objects.create(coords=(0, 0)) >>> foo.coords (0, 0) >>> foo.coords = (4, 2) >>> foo.coords.x # == foo.x == foo.coords[0] 4 Sidenote:: Subclassing the default implementation of composite field values should be easy: >>> type(foo.coords) >>> foo.coords.length 4.4721359549995796 >>> foo.coords += foo.velocity __ Johannes -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
I'd like to make one more pitch for a slightly different implementation here. My concern with CompositeField isn't based on the fact that it doesn't map one-to-one with a field in the table; it's that it doesn't have any of the semantics that are associated with a field. In particular, it can't be: - Assigned to. - Iterated over. - Or even have a value. My suggestion is to create an Index type that can be included in a class just like a field can. The example we've been using would then look like: class Foo(Model): x = models.FloatField() y = models.FloatField() a = models.ForeignKey(A) b = models.ForeignKey(B) coords = models.CompositeIndex((x, y)) pair = models.CompositeIndex((a, b), primary_key=True) We could have FieldIndex (the equivalent of the current db_index=True), CompositeIndex, and RawIndex, for things like expression indexes and other things that can be specified just as a raw SQL string. I think this is a much better contract to offer in the API than one based on field which would have to throw exceptions left and right for most of the common field operations. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Thu, Mar 17, 2011 at 09:33:43AM -0500, Jacob Kaplan-Moss wrote: > On Wed, Mar 16, 2011 at 4:24 AM, Johannes Dollinger > wrote: > > I would be nice if support for composite primary keys would be > > implemented as a special case of general composite fields. There > > would be no need for new Meta options: > > > > class Foo(Model): > > x = models.FloatField() > > y = models.FloatField() > > coords = models.CompositeField((x, y), db_index=True) > > a = models.ForeignKey(A) > > b = models.ForeignKey(B) > > pair = models.CompositeField((a, b), primary_key=True) > > > > A CompositeField descriptor would then return a namedtuple of its > > values and would support queries: > > > > filter(coords__x=42) > > filter(coords=(1,2)) > > > > Adding the individual fields may be optional, e.g, > > CompositeField((FloatField(), FloatField()), db_index=True). > > > > This has been proposed before: > > http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5 I must have overlooked this thread before... > I like this quite a bit. Of all the various syntaxes proposed here so > far, this is the first one that feels like it "fits" with the rest of > Django, and the first one I'm +1 on. I agree as well. This approach looks much cleaner from the design perspective. At least the syntax is more consistent than having the same information scattered throughout the individual fields and several Meta attributes in different cases. However, we'd either have a different API for specifying unique constraints for sets of fields than for composite keys or we'd have two options for the unique thing. (Or we'd lose backwards compatibility.) The current API could be simulated by creating an implicit CompositeField for each unique tuple with a reasonable name. A ForeignKey referencing a model with a primary CompositeField could then act as a CompositeField itself, creating implicit fields unless explicitly specified. Would this be a good idea? It would make it easier to mess with the values of the fields directly, possibly breaking the references to other rows, however, this is possible even with the way it is now. There is one thing though that's bothering me a little bit... At a first glance this looks to me like a lot more work than my original proposal. Now I'm not sure whether I should try to squeeze it all into a single project with the primary key support and ForeignKey and everything or rather just do the CompositeField with proper queryset support and save the primary keys for a later time. Thoughts? > I'm sensitive to Christophe's point that a "composite field" doesn't > map to a relational concept very well, but quite frankly that ship has > sailed. We've got ManyToManyFields, GenericForeignKeys, and once you > branch out into the ecosystem you find TagFields, PickleFields (ugh) > and so forth. Just a note, my first approach at the composite ForeignKey field would fall into this category anyway. I think we can't avoid that with this kind of functionality... Michal Petrucha signature.asc Description: Digital signature
Re: Composite primary keys
On 17 March 2011 16:18, Mike Axiak wrote: > Just to be clear, for this to be valid syntax doesn't this idea have to be > written as:: > class Foo(Model): > x = models.FloatField() > y = models.FloatField() > coords = models.CompositeField(('x', 'y'), db_index=True) > a = models.ForeignKey(A) > b = models.ForeignKey(B) > pair = models.CompositeField(('a', 'b'), primary_key=True) > (Note the quotes around the field names.) Actually, it works without the quotes as long as the fields are defined in the same class before the CompositeField: http://ideone.com/LPNzS -- Łukasz Rekucki -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
Just to be clear, for this to be valid syntax doesn't this idea have to be written as:: class Foo(Model): x = models.FloatField() y = models.FloatField() coords = models.CompositeField(('x', 'y'), db_index=True) a = models.ForeignKey(A) b = models.ForeignKey(B) pair = models.CompositeField(('a', 'b'), primary_key=True) (Note the quotes around the field names.) Not that it matters too much, but I think any discussion of syntax should have valid python. Cheers, Mike -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Wed, Mar 16, 2011 at 4:24 AM, Johannes Dollinger wrote: > I would be nice if support for composite primary keys would be implemented as > a special case of general composite fields. There would be no need for new > Meta options: > > class Foo(Model): > x = models.FloatField() > y = models.FloatField() > coords = models.CompositeField((x, y), db_index=True) > a = models.ForeignKey(A) > b = models.ForeignKey(B) > pair = models.CompositeField((a, b), primary_key=True) > > A CompositeField descriptor would then return a namedtuple of its values and > would support queries: > > filter(coords__x=42) > filter(coords=(1,2)) > > Adding the individual fields may be optional, e.g, > CompositeField((FloatField(), FloatField()), db_index=True). > > This has been proposed before: > http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5 I like this quite a bit. Of all the various syntaxes proposed here so far, this is the first one that feels like it "fits" with the rest of Django, and the first one I'm +1 on. I'm sensitive to Christophe's point that a "composite field" doesn't map to a relational concept very well, but quite frankly that ship has sailed. We've got ManyToManyFields, GenericForeignKeys, and once you branch out into the ecosystem you find TagFields, PickleFields (ugh) and so forth. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 16, 2011, at 9:13 AM, Carl Meyer wrote: > I'm not expressing an opinion one way or another on composite primary > key syntax, but I don't agree here that a Django model "field" must > map one-to-one to a database column. That's fair, but a composite index lacks some of the characteristics of a field (assignability, for example). Most DBs don't have functions that explicitly iterate over indexes, so such a thing isn't really readable, either. It might be appealing to have a models.Index base class that represents an index on a table, and have db_index=True be a shortcut to creating one. That might be more machinery than we want just for composite primary keys though. -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 16, 11:43 am, Christophe Pettus wrote: > On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote: > > > I would be nice if support for composite primary keys would be implemented > > as a special case of general composite fields. > > It's appealing, but the reality is that no existing back-end actually has > such an animal as a composite field. In all of these cases, what we're > really creating is a composite index on a set of standard fields. > Introducing a more powerful index-creation syntax into Django isn't a bad > idea, but we shouldn't call it a "field" if it is not. I'm not expressing an opinion one way or another on composite primary key syntax, but I don't agree here that a Django model "field" must map one-to-one to a database column. It already does not, in the case of ManyToManyField, and at some point I would like to introduce (irrespective of composite primary keys) a more general ORM abstraction for composite fields (i.e. model Fields that map to more than one database column) as a path to cleaning up the implementation of GenericForeignKey. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote: > I would be nice if support for composite primary keys would be implemented as > a special case of general composite fields. It's appealing, but the reality is that no existing back-end actually has such an animal as a composite field. In all of these cases, what we're really creating is a composite index on a set of standard fields. Introducing a more powerful index-creation syntax into Django isn't a bad idea, but we shouldn't call it a "field" if it is not. -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
I would be nice if support for composite primary keys would be implemented as a special case of general composite fields. There would be no need for new Meta options: class Foo(Model): x = models.FloatField() y = models.FloatField() coords = models.CompositeField((x, y), db_index=True) a = models.ForeignKey(A) b = models.ForeignKey(B) pair = models.CompositeField((a, b), primary_key=True) A CompositeField descriptor would then return a namedtuple of its values and would support queries: filter(coords__x=42) filter(coords=(1,2)) Adding the individual fields may be optional, e.g, CompositeField((FloatField(), FloatField()), db_index=True). This has been proposed before: http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5 __ Johannes -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Wed, 16 Mar 2011 07:49:17 +0200, Michal Petrucha wrote: On Tue, Mar 15, 2011 at 09:15:29PM -0500, Javier Guerra Giraldez wrote: On Tue, Mar 15, 2011 at 7:14 PM, Christophe Pettus wrote: > A concern here is that composite indexes, like unique, are > sensitive to the ordering of the fields, which means that the > ordering of the fields in the class declaration becomes important. This is the same reason a new Meta flag has been agreed upon in the past. (That, however, does not mean it has to be that way.) a simplistic proposal: the order of the fields on a composite index is determined by the exact value given to the primary_key argument. that way, just setting primary_key=True on a few fields won't guarantee order, but something like: class City (models.Model): country = models.CharField (max_length=2, primary_key=1) state = models.CharField (max_length=2, primary_key=2) city = models.CharField (max_length=3, primary_key=2) Name = models.CharField (max_length=20) would set the (country,state,city) primary key in the obvious order. in short: giving any non-falsy value to primary_key would add the field to the key, the exact value would determine ordering. I like this proposal. It might even be easier to implement than fiddling with new Meta flags and everything. One minor detail though, just setting primary_key=True on multiple fields would still have to guarantee some ordering since the primary key would be represented by a tuple. If you don't know for sure in which order the values are, you can't really use the .pk property. This feels like something that wants a named-tuple (or a full blown dict). Alternatively, provide a method on the model class that takes the name=value arguments (queryset style) and returns the right pk tuple. Otherwise, the exact ordering of the fields in the pk tuple becomes yet another implicit(!) part of the model's contract - and any code that wants to use this model will be that much more brittle. This would require a thorough explanation in the docs, that if you supply values that compare equal to primary_key, the order of fields will be the same as in the model definition. Michal Petrucha -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Tue, Mar 15, 2011 at 09:15:29PM -0500, Javier Guerra Giraldez wrote: > On Tue, Mar 15, 2011 at 7:14 PM, Christophe Pettus wrote: > > A concern here is that composite indexes, like unique, are > > sensitive to the ordering of the fields, which means that the > > ordering of the fields in the class declaration becomes important. This is the same reason a new Meta flag has been agreed upon in the past. (That, however, does not mean it has to be that way.) > a simplistic proposal: > > the order of the fields on a composite index is determined by the > exact value given to the primary_key argument. > > that way, just setting primary_key=True on a few fields won't > guarantee order, but something like: > > class City (models.Model): > country = models.CharField (max_length=2, primary_key=1) > state = models.CharField (max_length=2, primary_key=2) > city = models.CharField (max_length=3, primary_key=2) > Name = models.CharField (max_length=20) > > would set the (country,state,city) primary key in the obvious order. > > in short: giving any non-falsy value to primary_key would add the > field to the key, the exact value would determine ordering. I like this proposal. It might even be easier to implement than fiddling with new Meta flags and everything. One minor detail though, just setting primary_key=True on multiple fields would still have to guarantee some ordering since the primary key would be represented by a tuple. If you don't know for sure in which order the values are, you can't really use the .pk property. This would require a thorough explanation in the docs, that if you supply values that compare equal to primary_key, the order of fields will be the same as in the model definition. Michal Petrucha signature.asc Description: Digital signature
Re: Composite primary keys
On Tue, Mar 15, 2011 at 7:14 PM, Christophe Pettus wrote: > A concern here is that composite indexes, like unique, are sensitive to the > ordering of the fields, which means that the ordering of the fields in the > class declaration becomes important. a simplistic proposal: the order of the fields on a composite index is determined by the exact value given to the primary_key argument. that way, just setting primary_key=True on a few fields won't guarantee order, but something like: class City (models.Model): country = models.CharField (max_length=2, primary_key=1) state = models.CharField (max_length=2, primary_key=2) city = models.CharField (max_length=3, primary_key=2) Name = models.CharField (max_length=20) would set the (country,state,city) primary key in the obvious order. in short: giving any non-falsy value to primary_key would add the field to the key, the exact value would determine ordering. -- Javier -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 15, 2011, at 5:06 PM, Russell Keith-Magee wrote: > And if you mark > multiple fields, then you have a composite primary key composed of > those fields. A concern here is that composite indexes, like unique, are sensitive to the ordering of the fields, which means that the ordering of the fields in the class declaration becomes important. That could, potentially, be surprising. -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Tue, Mar 15, 2011 at 11:11 PM, Andrew Godwin wrote: > On 14/03/11 21:14, Michal Petrucha wrote: >> >> Good evening (or whatever it is in everyone's timezone). >> >> I'm an undergrad computer science student at the Faculty of >> Mathematics, Physics and Informatics, Commenius University, >> Bratislava, Slovakia and I'm willing to participate in this year's >> GSoc. I'm interested in fixing the six-year-old open ticket in trac >> concerning the subject, http://code.djangoproject.com/ticket/373 > > Firstly, thanks for proposing a GSoC project - it's always nice to have more > students. Bear in mind that the list of accepted organisations hasn't been > published yet, but we're pretty hopeful Django will make it in again this > year. I'd like to second what Andrew has said. This is definitely a project with enough meat to fill a GSoC, and it's something that has been on Django's wish list for a long time. I also agree with Andrew's answers to your questions, with one exception: >> - The composite primary key would be specified as a tuple of strings >> in a primary_key attribute inside a model's Meta class instead of >> having a field with primary_key=True. > > That seems to match with our current handling of things like unique, so that > seems fairly reasonable. The difference between primary_key and unique is that you can have multiple unique conditions, including a number of individual field uniques *plus* a series of grouped uniques. However you can only have one primary key field (or field set). I'd like to suggest an alternate representation. At present, you can make any field a primary key by setting primary_key=True on that field. The Meta class then validates to ensure that there is only one field that has that attribute set (and inserts an AutoField if no field has it set). Another way to represent primary keys would be to relax this validation constraint. If you mark no field as primary_key=True, then an AutoField is added. If you mark one field, then that field is the primary key. And if you mark multiple fields, then you have a composite primary key composed of those fields. This removes the need to introduce a new Meta flag, but more importantly, it means you won't have to resolve discrepancies between fields with primary_key and a Meta.primary_key option. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On 14/03/11 21:14, Michal Petrucha wrote: Good evening (or whatever it is in everyone's timezone). I'm an undergrad computer science student at the Faculty of Mathematics, Physics and Informatics, Commenius University, Bratislava, Slovakia and I'm willing to participate in this year's GSoc. I'm interested in fixing the six-year-old open ticket in trac concerning the subject, http://code.djangoproject.com/ticket/373 Firstly, thanks for proposing a GSoC project - it's always nice to have more students. Bear in mind that the list of accepted organisations hasn't been published yet, but we're pretty hopeful Django will make it in again this year. Before I dig deeper into the issue I would like to know whether there is interest in having this fixed and whether it is worth a full GSoC project. Also, is there already any work regarding this issue underway? If so, would it be reasonable for me to go on with this project? Anyway, if I'm to take on the task, there are quite a few design considerations to be taken care of. For starters, I've read through David Cramer's work on this from two-three years ago. I'd stick to the API skeleton that was agreed on back then, however, there remain lots of other unresolved questions. The following list is in no way meant to be exhaustive, it's just a list of things that came to my mind during the past hour. In fact, I'd appreciate other issues that would need to be kept in mind I forgot to mention. - The composite primary key would be specified as a tuple of strings in a primary_key attribute inside a model's Meta class instead of having a field with primary_key=True. That seems to match with our current handling of things like unique, so that seems fairly reasonable. - The pk property of model instances would be a tuple instead of a single value for composite key models. So the property will vary type based on the model? Obviously this is needed for backwards-compatability, but it's still going to affect e.g. library authors, so we'd need to make sure everything that previously accepted a PK also accepts a tuple. - The admin could reference composite keys using some kind of smart string escaping, for example escaping the , (comma) and using it as a delimiter. That should work, though it might be a bit ugly; again, I can't think of a better alternative. - This could maybe even be used in generic relations. Is it reasonable to support this in generic relations? The following post suggests it might not be the best idea: http://groups.google.com/group/django-developers/msg/dea0e360c6cd37a6 Generic relations are ugly and inefficient as it is; I'm not sure that including them in something like this is strictly necessary, much as Malcolm suggests, and probably outside the scope of a GSoC project (something of this scale is already touching a lot of places in the model layer as it is) - The managers and querysets would have to be updated to handle composite primary keys correctly. - Consequently, there would need to be added support in the SQL compiler. - The same holds for syncdb, inspectdb would be also nice. I'd say inspectdb isn't strictly necessary - not very many people use it. I'll also be looking at potentally merging in some schema alteration code (i.e. moving parts of the South database backends) over the next few months, so it's going to affect that, too, though I'm more than happy to help fix that part (the existing creation code will definitely need changing, at least). - ForeignKeys would have to be backed by multiple database columns. How should they be named by default? How should their names be overriden? Should db_column expect a tuple of strings? Should there be another db_column_prefix option to prefix the names with a common string? These are some very good questions, and something we need a good discussion here about; I'd personally say doing "fkname_remotecolname" is best for the columns (which then means "author_id" still matches most DBs, and you could also have something like "passport_country_id, passport_number"). I'm also -0 on the idea of a db_column_prefix, but db_column is going to need to take tuples/sequences for multi-column FKs. - What about the ForeignKey field's attname? Should it be a tuple of names or should it be a single string pointing to a tuple of attributes? I'd say a single string pointing to a tuple - I don't like fields which 'magically' make more attributes than the name they were assigned. That also makes sense if you think of the foreign key as a single entity composed of several values. - How should a person trying to add a ForeignKey field pointing to its own model into the primary_key
Composite primary keys
Good evening (or whatever it is in everyone's timezone). I'm an undergrad computer science student at the Faculty of Mathematics, Physics and Informatics, Commenius University, Bratislava, Slovakia and I'm willing to participate in this year's GSoc. I'm interested in fixing the six-year-old open ticket in trac concerning the subject, http://code.djangoproject.com/ticket/373 Before I dig deeper into the issue I would like to know whether there is interest in having this fixed and whether it is worth a full GSoC project. Also, is there already any work regarding this issue underway? If so, would it be reasonable for me to go on with this project? Anyway, if I'm to take on the task, there are quite a few design considerations to be taken care of. For starters, I've read through David Cramer's work on this from two-three years ago. I'd stick to the API skeleton that was agreed on back then, however, there remain lots of other unresolved questions. The following list is in no way meant to be exhaustive, it's just a list of things that came to my mind during the past hour. In fact, I'd appreciate other issues that would need to be kept in mind I forgot to mention. - The composite primary key would be specified as a tuple of strings in a primary_key attribute inside a model's Meta class instead of having a field with primary_key=True. - The pk property of model instances would be a tuple instead of a single value for composite key models. - The admin could reference composite keys using some kind of smart string escaping, for example escaping the , (comma) and using it as a delimiter. - This could maybe even be used in generic relations. Is it reasonable to support this in generic relations? The following post suggests it might not be the best idea: http://groups.google.com/group/django-developers/msg/dea0e360c6cd37a6 - The managers and querysets would have to be updated to handle composite primary keys correctly. - Consequently, there would need to be added support in the SQL compiler. - The same holds for syncdb, inspectdb would be also nice. - ForeignKeys would have to be backed by multiple database columns. How should they be named by default? How should their names be overriden? Should db_column expect a tuple of strings? Should there be another db_column_prefix option to prefix the names with a common string? - What about the ForeignKey field's attname? Should it be a tuple of names or should it be a single string pointing to a tuple of attributes? - How should a person trying to add a ForeignKey field pointing to its own model into the primary_key be punished? - Does it make sense to make a subset of columns created by a ForeignKey part of a primary key? - The forms framework would need a way to pass composite ForeignKeys as parameters. - What about OneToOne? Some items in this list are my ideas on how to implement something, some are things already decided during previous attempts, others are questions which I believe are not entirely up to me to decide on. I'll be grateful for any comments on any of these points and as I said earlier, any other considerations related to the topic. Also, these would somehow have to be split into two parts: those to be focused on from the perspective of this project and those to be postponed to a later stage or with little relevance. Thanks in advance for any feedback. Michal Petrucha signature.asc Description: Digital signature
Re: Composite Primary Keys
Well GenericRelation's or whatever are an attempt to provide some composite space, but we need an expandable space, not something tied to content type's. We need to say mykey = CompositeField(field, field, field) or something. No one wanted to discuss it at DjangoCon, so no. On Thu, Oct 30, 2008 at 3:57 PM, Joey Wilhelm <[EMAIL PROTECTED]> wrote: > That would be great. The project I am working on now won't be doing > anything too terribly complex just yet; I mainly need the admin support to > make my life a little easier. > > As to the API, I saw several proposals earlier along on this thread, but > obviously nothing solid. Did anything ever come from DjangoCon on this > topic? What issues still need to be addressed in this design? > > On Thu, Oct 30, 2008 at 13:46, David Cramer <[EMAIL PROTECTED]> wrote: > >> It allows you to use them, automatically creates them, and has some of the >> admin handling done. However, there's still no API design around >> multi-column fields (no one seems to want to talk about it) so I'm pretty >> much stopped working on it. >> >> e.g. You can't say field1 = this, field2 = that, and then say compositekey >> = field1,field2 you instead are forced to do key1=blah, key2=blah in all >> your lookups, and no easy foreignkey properties. >> >> I'm running this on production environments, so it works fine, but I can >> up SVN and fix any conflicts and post a patch again. >> >> -- >> David Cramer >> Director of Technology >> iBegin >> http://www.ibegin.com/ >> >> >> > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
That would be great. The project I am working on now won't be doing anything too terribly complex just yet; I mainly need the admin support to make my life a little easier. As to the API, I saw several proposals earlier along on this thread, but obviously nothing solid. Did anything ever come from DjangoCon on this topic? What issues still need to be addressed in this design? On Thu, Oct 30, 2008 at 13:46, David Cramer <[EMAIL PROTECTED]> wrote: > It allows you to use them, automatically creates them, and has some of the > admin handling done. However, there's still no API design around > multi-column fields (no one seems to want to talk about it) so I'm pretty > much stopped working on it. > > e.g. You can't say field1 = this, field2 = that, and then say compositekey > = field1,field2 you instead are forced to do key1=blah, key2=blah in all > your lookups, and no easy foreignkey properties. > > I'm running this on production environments, so it works fine, but I can up > SVN and fix any conflicts and post a patch again. > > -- > David Cramer > Director of Technology > iBegin > http://www.ibegin.com/ > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
It allows you to use them, automatically creates them, and has some of the admin handling done. However, there's still no API design around multi-column fields (no one seems to want to talk about it) so I'm pretty much stopped working on it. e.g. You can't say field1 = this, field2 = that, and then say compositekey = field1,field2 you instead are forced to do key1=blah, key2=blah in all your lookups, and no easy foreignkey properties. I'm running this on production environments, so it works fine, but I can up SVN and fix any conflicts and post a patch again. On Thu, Oct 30, 2008 at 3:40 PM, Joey Wilhelm <[EMAIL PROTECTED]> wrote: > > David, > > What is the current status of this patch? I'm starting up a new > project which pretty much desperately needs this support as well. I > could work around it, but the thought of adding AutoFields to all of > these models which really -do not- need them, makes me a bit ill. > > I would be more than willing to help test your implementation if there > is anything usable yet. This is one of the pieces that's getting me > all twitchy waiting for it. > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
David, What is the current status of this patch? I'm starting up a new project which pretty much desperately needs this support as well. I could work around it, but the thought of adding AutoFields to all of these models which really -do not- need them, makes me a bit ill. I would be more than willing to help test your implementation if there is anything usable yet. This is one of the pieces that's getting me all twitchy waiting for it. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
What we hope to achieve here is full support within the Django core, as opposed to specifying some kind of model. The only thing I have left to do is implement composite fields, but there's still no API for it. On Sat, Oct 4, 2008 at 12:03 PM, Eric <[EMAIL PROTECTED]> wrote: > > Hi, > i just discover this thread, I am working on this problem; you may > take a look at > http://kenai.com/projects/django-trac/pages/LegacyModule > > legacy is a module in my "django hacks trac" (or djac) project; it > aims to deal with > tables with no primary key or with composite pk. It provides 2 > methods: > > - use of oid field (works on sqlite, oracle, postgres <= 8) > - composite pk (for mysql that provides no oid field) > > cheers, > Eric > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
Hi, i just discover this thread, I am working on this problem; you may take a look at http://kenai.com/projects/django-trac/pages/LegacyModule legacy is a module in my "django hacks trac" (or djac) project; it aims to deal with tables with no primary key or with composite pk. It provides 2 methods: - use of oid field (works on sqlite, oracle, postgres <= 8) - composite pk (for mysql that provides no oid field) cheers, Eric --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
On Wed, 2008-09-03 at 23:22 -0700, David Cramer wrote: > This is one of those things that I really need to get hammered out > (for our platform). I'm willing to do all the work, but I need a > design around it to where the patch won't get rejected :) No question. And for that, we have this very handy mailing list with archives that will store all the discussion. Meanwhile, this weekend, we have a two day conference with a very full schedule, no scheduled downtime and a limited attendance group. This is all at the end of a long process of nothing but dawn to dusk bug fixing for a group of us, so some of us will be relaxing. So by all means have a conversation with some people, that's a given. But you'll still need to bring the results to the mailing list in a coherent format so that it can be looked at when we start to consider the patch. This is a serious proposal. It's being taken seriously. But realise the broader things going on as well. If you're hoping/expecting to get this resolved in a couple of days, you are, frankly, optimistic. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
This is one of those things that I really need to get hammered out (for our platform). I'm willing to do all the work, but I need a design around it to where the patch won't get rejected :) On Sep 3, 11:08 am, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Tue, 2008-09-02 at 22:02 -0700, David Cramer wrote: > > For anyone who's interested, it'd be great to meetup at DjangoCon to > > go over a good design approach to composite fields. > > Take notes. There's going to be a lot going on at DjangoCon (including > celebrating), so there will be a group who are putting off thinking > about this until quieten down who'll likely be busy throughout the > weekend. In a few weeks we can start looking at this more concretely on > the list and working out how it might fit into the whole. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
For anyone who's interested, it'd be great to meetup at DjangoCon to go over a good design approach to composite fields. On Sep 2, 2:44 pm, Rock <[EMAIL PROTECTED]> wrote: > One use case for Composite Primary Keys is for setting up database > partitions. In my case I am using Range-Hash partitions with the range > determined by an IntegerField called "ISOweek" and the hash working > off of the "id" field supplied by Django. To allow this partitioning > to work, the primary key must be a composite primary key incorporating > the "ISOweek" and the "id" fields. My versions of the sqlreset and > reset management functions do this while also ensuring that "id" is > marked as unique even though it is not the primary key. This allows a > ForeignKey pointed at my partitioned model to work correctly by > setting "id" as the to_field. (If "id" is not set as unique, Django > and/or the database will fail in its' attempt to set up the full > foreign key relationship.) > > The initial version of Composite Primary Keys should not preclude this > scenario, however full support for setting up and managing partitioned > models need not be included at this time. (I plan to help add that > later.) The interesting point is that support for related fields for > the Composite Primary Key is not required in order to support this > particular use case. > > Rock > > On Aug 28, 8:05 pm, "David Cramer" <[EMAIL PROTECTED]> wrote: > > > I'm not quite sure how that relates to Composite Primary Keys? > > > A ForeignKey would point to multiple internal fields, but it should look > > like it's a single field. At the same time, this would open up the > > possibility for Composite Foreign Keys, which would mean it could point to > > multiple public fields. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
On Tue, 2008-09-02 at 22:02 -0700, David Cramer wrote: > For anyone who's interested, it'd be great to meetup at DjangoCon to > go over a good design approach to composite fields. Take notes. There's going to be a lot going on at DjangoCon (including celebrating), so there will be a group who are putting off thinking about this until quieten down who'll likely be busy throughout the weekend. In a few weeks we can start looking at this more concretely on the list and working out how it might fit into the whole. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
One use case for Composite Primary Keys is for setting up database partitions. In my case I am using Range-Hash partitions with the range determined by an IntegerField called "ISOweek" and the hash working off of the "id" field supplied by Django. To allow this partitioning to work, the primary key must be a composite primary key incorporating the "ISOweek" and the "id" fields. My versions of the sqlreset and reset management functions do this while also ensuring that "id" is marked as unique even though it is not the primary key. This allows a ForeignKey pointed at my partitioned model to work correctly by setting "id" as the to_field. (If "id" is not set as unique, Django and/or the database will fail in its' attempt to set up the full foreign key relationship.) The initial version of Composite Primary Keys should not preclude this scenario, however full support for setting up and managing partitioned models need not be included at this time. (I plan to help add that later.) The interesting point is that support for related fields for the Composite Primary Key is not required in order to support this particular use case. Rock On Aug 28, 8:05 pm, "David Cramer" <[EMAIL PROTECTED]> wrote: > I'm not quite sure how that relates to Composite Primary Keys? > > A ForeignKey would point to multiple internal fields, but it should look > like it's a single field. At the same time, this would open up the > possibility for Composite Foreign Keys, which would mean it could point to > multiple public fields. > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
I'm not quite sure how that relates to Composite Primary Keys? A ForeignKey would point to multiple internal fields, but it should look like it's a single field. At the same time, this would open up the possibility for Composite Foreign Keys, which would mean it could point to multiple public fields. On Thu, Aug 28, 2008 at 5:36 PM, Rock <[EMAIL PROTECTED]> wrote: > > To be clear, the syntax is: > > myfkey = models.ForeignKey(SomeClass,to_field="id") > > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
To be clear, the syntax is: myfkey = models.ForeignKey(SomeClass,to_field="id") --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
Ha! It turns out that a to_field option already exists for ForeignKey. (I did not know that yesterday.) I have just verified that to_field(SomeClass,"id") works fine even if the PRIMARY KEY uses multiple columns. However, and this is the key point, the id field has to be marked as UNIQUE. To prove all of this I have created a sqlresetpartition and resetpartition command that looks for classes to partition within your settings. It sets up the required multi-column primary key, marks the ID field as UNIQUE, and also does the other magic to create partitioned tables. (It also creates partitioned indexes as required.) This only works for Oracle and the solution is special cased for my needs, but it points the way forward. I will look over the partitioning logic for MySQL when I get a chance. Maybe by Django 1.5 or so we can include direct support for partitioned models. BTW, once I have the models properly created with partitions, none of my other django code requires any changes at all. Besides improved performance, I must say that it is a thrill to delete hundreds of thousands of old rows in less than a second. This is the key feature I needed for my django app. On Aug 27, 6:27 pm, Rock <[EMAIL PROTECTED]> wrote: > Well for one thing, if one of the columns happens to be named "ID", we > should use that for the relatedfields lookup column and that is that. > (BTW, does your approach allow the Django supplied ID field to be > combined with some other field(s) to make a multi-column key? This > would be bang up for future partitioning support.) > > Next I would suggest adding a meta model column designation like > "id_field" to specify a field to use for related classes. This might > be a good "80/20" solution that could serve for an initial test > version. > > On Aug 27, 5:27 pm, "David Cramer" <[EMAIL PROTECTED]> wrote: > > > > > Really I'm stuck at an architectural point. > > > I have database validation and synchronization done, and the admin is > > working. > > > What is left is more or less handling relatedfield lookups. The issue is, > > that field's are designed to reference more than one field, so it's a tough > > design deicision to make on how that should be approached.- Hide quoted > > text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
Well for one thing, if one of the columns happens to be named "ID", we should use that for the relatedfields lookup column and that is that. (BTW, does your approach allow the Django supplied ID field to be combined with some other field(s) to make a multi-column key? This would be bang up for future partitioning support.) Next I would suggest adding a meta model column designation like "id_field" to specify a field to use for related classes. This might be a good "80/20" solution that could serve for an initial test version. On Aug 27, 5:27 pm, "David Cramer" <[EMAIL PROTECTED]> wrote: > Really I'm stuck at an architectural point. > > I have database validation and synchronization done, and the admin is > working. > > What is left is more or less handling relatedfield lookups. The issue is, > that field's are designed to reference more than one field, so it's a tough > design deicision to make on how that should be approached. > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
What I had briefly discussed with malcom was using ordered tuples but switching up the defaults to use actualy field lookups. MyModel.objects.get(pk=(1, 2)) or MyModel.objects.get(foo=1, bar=2) If we could come up with some design for multi-column fields I'm wiling to put in the work. On Wed, Aug 27, 2008 at 6:15 PM, Alberto García Hierro <[EMAIL PROTECTED]>wrote: > > > El 28/08/2008, a las 0:27, David Cramer escribió: > > > Really I'm stuck at an architectural point. > > > > I have database validation and synchronization done, and the admin > > is working. > > > > What is left is more or less handling relatedfield lookups. The > > issue is, that field's are designed to reference more than one > > field, so it's a tough design deicision to make on how that should > > be approached. > > I think the best (and the only one right) solution involves adding > multicolumn fields to Django and doing lookups with some syntax like > Model.objects.get(pk=('foo', 1)). There are other hackish approaches, > like using hash(tuple(pk[0], pk[1], ..., pk[n])) as foreign key, that > could work. However, I won't rely on them, since I'm not sure if > hash() implementation is guaranteed to be kept as is. > > On other related point, what's the status of multicolumn fields? > > Regards, > Alberto > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
El 28/08/2008, a las 0:27, David Cramer escribió: > Really I'm stuck at an architectural point. > > I have database validation and synchronization done, and the admin > is working. > > What is left is more or less handling relatedfield lookups. The > issue is, that field's are designed to reference more than one > field, so it's a tough design deicision to make on how that should > be approached. I think the best (and the only one right) solution involves adding multicolumn fields to Django and doing lookups with some syntax like Model.objects.get(pk=('foo', 1)). There are other hackish approaches, like using hash(tuple(pk[0], pk[1], ..., pk[n])) as foreign key, that could work. However, I won't rely on them, since I'm not sure if hash() implementation is guaranteed to be kept as is. On other related point, what's the status of multicolumn fields? Regards, Alberto --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
Really I'm stuck at an architectural point. I have database validation and synchronization done, and the admin is working. What is left is more or less handling relatedfield lookups. The issue is, that field's are designed to reference more than one field, so it's a tough design deicision to make on how that should be approached. On Wed, Aug 27, 2008 at 5:23 PM, Rock <[EMAIL PROTECTED]> wrote: > > Any progress on this patch David? I would be happy to take a look at > whatever you have and perhaps help out with completing the patch. > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
Any progress on this patch David? I would be happy to take a look at whatever you have and perhaps help out with completing the patch. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
On Thu, May 8, 2008 at 7:57 PM, David Cramer <[EMAIL PROTECTED]> wrote: > I swear I saw something about work being done on this. Has anyone > begun? If not I'll gladly throw up a patch to get it into trunk. Please do. If you make sure it solves all three problems I outlined in #373 (http://code.djangoproject.com/ticket/373#comment:3) I'll make sure it gets in before the next release. Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Composite Primary Keys
On Fri, May 9, 2008 at 8:57 AM, David Cramer <[EMAIL PROTECTED]> wrote: > > I swear I saw something about work being done on this. Has anyone > begun? If not I'll gladly throw up a patch to get it into trunk. The topic comes up fairly regularly, but I'm not aware of anyone that is actively working on it. The most recent reference I am aware of is this one where it was proposed as a GSOC project; the comment from Malcolm is the most enlightening: http://groups.google.com/group/django-developers/browse_thread/thread/647009a9a4439f0b Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Composite Primary Keys
I swear I saw something about work being done on this. Has anyone begun? If not I'll gladly throw up a patch to get it into trunk. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---