Re: Django & memcache hashing

2008-11-19 Thread Malcolm Tredinnick


On Thu, 2008-11-20 at 07:20 +0300, Ivan Sagalaev wrote:
> Malcolm Tredinnick wrote:
> > Okay. If we go this path, it's something to include in Django, rather
> > than recommending yet another caching package. We either make it a
> > configuration option to force python-memcache or cmemcache or we just
> > "Do The Right Thing", with the latter being preferable.
> 
> What concerns me is that this will break the usage of memcached without 
> Django's cache API. I had the need a couple of times to do plain 
> instantiation of memcache.Client and work with it. If it won't see the 
> cache the same way as Django does it would be that very issue, hard to 
> debug, that started this thread.

This is indeed a concern. I was intending to put in a module that you
can import to get the same behaviour as Django. So instead of

import memcached

you write

from django.core.cache import memcached

I'm not 100% certain, though, that this is the way to go. I'm letting it
bounce around for a few days. Both options have their drawbacks and it's
kind of a matter of weighing up which inconvenience is more likely to
occur, given that they're both relatively uncommon (after all, if you're
accessing Django objects via direct usage, you need to be using Django's
get_cache_key() and the like anyway).

Regards,
Malcolm



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread Ivan Sagalaev

Malcolm Tredinnick wrote:
> Okay. If we go this path, it's something to include in Django, rather
> than recommending yet another caching package. We either make it a
> configuration option to force python-memcache or cmemcache or we just
> "Do The Right Thing", with the latter being preferable.

What concerns me is that this will break the usage of memcached without 
Django's cache API. I had the need a couple of times to do plain 
instantiation of memcache.Client and work with it. If it won't see the 
cache the same way as Django does it would be that very issue, hard to 
debug, that started this thread.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: order_by with distinct behavior in orm

2008-11-19 Thread Steve Holden

Malcolm Tredinnick wrote:
[...]
> Hopefully, MySQL will fix their bug one day. That's the way to resolve
> this.
> 
Just so long as nobody holds their breath waiting ...

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



RE: order_by with distinct behavior in orm

2008-11-19 Thread Trivedi, Apaar

Ok, I understand what you are saying now, and I hadn't thought about it
the way you present it.  Thanks.

Apaar


-Original Message-
From: django-developers@googlegroups.com
[mailto:[EMAIL PROTECTED] On Behalf Of Malcolm
Tredinnick
Sent: Wednesday, November 19, 2008 7:00 PM
To: django-developers@googlegroups.com
Subject: Re: order_by with distinct behavior in orm



On Wed, 2008-11-19 at 15:41 -0800, Apaar Trivedi wrote:
[...]
> But how about at least introducing some way to suppress the order_by
> column being added to the select fields?  This way postgres is still
> supported, and mysql distinct will work as well.

I'm strongly against this. We should be following correct SQL practices
and SQL requires the ordering columns in the select list (the output
columns). This is actually a *good* thing, too. If you're ordering by
something that has multiple entries for a given set of output columns
(output columns without the ordering columns, in this case), there's
actually no correct answer when you add a distinct() qualifier, since
there's no well-defined choice for which of the many possible orderings
you choose in that case -- the same result tuple could appear first,
least of somewhere in the middle, purely based on which completely
arbitrary choice is made. Thus, the behaviour you're asking for means
ordering is not a well-defined operation.

As much as possible, Django provides a uniform interface over databases
and, in this case, MySQL is doing something that we really shouldn't be
encouraging, so special-casing it would not be encouraging best
practices.

Hopefully, MySQL will fix their bug one day. That's the way to resolve
this.

Regards,
Malcolm




--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread Ludvig Ericson

On Nov 20, 2008, at 00:55, Malcolm Tredinnick wrote:
> Okay. If we go this path, it's something to include in Django, rather
> than recommending yet another caching package. We either make it a
> configuration option to force python-memcache or cmemcache or we just
> "Do The Right Thing", with the latter being preferable.

Absolutely -- that'd be my most favored solution as at that point it
wouldn't matter if you ran cmemcache or python-memcached (other than
that cmemcache might kill the process it runs in on protocol error.)

So, do you want me to make a patch or could you do it? I feel I'm not
entirely sure exactly where it should reside and so forth. But I could
take the time and find out if you're too busy.

> I hadn't realised from your earlier description that you had a
> non-intrusive change that we could drop into Django. My
> misunderstanding. Glad we're on the same page now.


I'm not a native speaker, so I might not be expressing my self in the
best English you've ever seen.

- Ludvig

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: order_by with distinct behavior in orm

2008-11-19 Thread Malcolm Tredinnick


On Wed, 2008-11-19 at 15:41 -0800, Apaar Trivedi wrote:
[...]
> But how about at least introducing some way to suppress the order_by
> column being added to the select fields?  This way postgres is still
> supported, and mysql distinct will work as well.

I'm strongly against this. We should be following correct SQL practices
and SQL requires the ordering columns in the select list (the output
columns). This is actually a *good* thing, too. If you're ordering by
something that has multiple entries for a given set of output columns
(output columns without the ordering columns, in this case), there's
actually no correct answer when you add a distinct() qualifier, since
there's no well-defined choice for which of the many possible orderings
you choose in that case -- the same result tuple could appear first,
least of somewhere in the middle, purely based on which completely
arbitrary choice is made. Thus, the behaviour you're asking for means
ordering is not a well-defined operation.

As much as possible, Django provides a uniform interface over databases
and, in this case, MySQL is doing something that we really shouldn't be
encouraging, so special-casing it would not be encouraging best
practices.

Hopefully, MySQL will fix their bug one day. That's the way to resolve
this.

Regards,
Malcolm


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread Malcolm Tredinnick


On Thu, 2008-11-20 at 00:46 +0100, Ludvig Ericson wrote:
> On Nov 20, 2008, at 00:11, Malcolm Tredinnick wrote:
> > That's possible. The hash function in the pure-Python memcached  
> > wrapper
> > is replaceable (it's an attribute), so I was looking at replacing it
> > with the version from cmemcached. Using a third hashing algorithm  
> > would
> > be a bit silly, since then there's no compatibility with anything else
> > and it's just extra effort for no huge gain.
> 
> And that's _exactly_ what my cmemcache_hash package does! :-)
> 
> http://pypi.python.org/pypi/cmemcache_hash
> 
> I don't know if I attached a license, but it's BSD - so feel free to rip
> it.

Okay. If we go this path, it's something to include in Django, rather
than recommending yet another caching package. We either make it a
configuration option to force python-memcache or cmemcache or we just
"Do The Right Thing", with the latter being preferable.

I hadn't realised from your earlier description that you had a
non-intrusive change that we could drop into Django. My
misunderstanding. Glad we're on the same page now.

Regards,
Malcolm



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



order_by with distinct behavior in orm

2008-11-19 Thread Apaar Trivedi

I know this is a topic you all may have discussed a number of times,
but I feel I need to bring it up still.  I noticed that when doing an
order_by, the column name is implicitly added to the select.  However,
this causes a problem when doing a distinct, as well.

Consider that someone is selecting entries and ordering by a date, but
they also wish to only get unique entries.  One would think that doing
a distinct would solve this, but in fact it does not.  When the
order_by is done, the date is added to the select, and this causes a
problem in mysql, because now I am not getting distinct entries, and
duplicate entries with different dates associated with them would
appear.  I am sure I don't need to explain this too much as I know
this has been discussed and is even provided as a caveat in the
documentation.

I also see this ticket: http://code.djangoproject.com/ticket/7142,
which explains that postgres cannot do an order_by without the column
name present in the select field, so I can understand this behavior.

But how about at least introducing some way to suppress the order_by
column being added to the select fields?  This way postgres is still
supported, and mysql distinct will work as well.

thanks
Apaar

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread Ludvig Ericson

On Nov 20, 2008, at 00:11, Malcolm Tredinnick wrote:
> That's possible. The hash function in the pure-Python memcached  
> wrapper
> is replaceable (it's an attribute), so I was looking at replacing it
> with the version from cmemcached. Using a third hashing algorithm  
> would
> be a bit silly, since then there's no compatibility with anything else
> and it's just extra effort for no huge gain.

And that's _exactly_ what my cmemcache_hash package does! :-)

http://pypi.python.org/pypi/cmemcache_hash

I don't know if I attached a license, but it's BSD - so feel free to rip
it.

- Ludvig

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread Malcolm Tredinnick


On Wed, 2008-11-19 at 03:26 -0800, taleinat wrote:
[...]
> What about having Django's memcached cache backend implement its own
> hashing algorithm? This could be the "standard" one used in
> libmemcache and cmemcache, or perhaps a consistent hashing algorithm
> [1] such as libketama[2].

That's possible. The hash function in the pure-Python memcached wrapper
is replaceable (it's an attribute), so I was looking at replacing it
with the version from cmemcached. Using a third hashing algorithm would
be a bit silly, since then there's no compatibility with anything else
and it's just extra effort for no huge gain.

Regards,
Malcolm


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Using a HTML to fire off a data processing script on the server (REST or SOAP)

2008-11-19 Thread Steve Holden

Shao:

This is a very general question, and is only indirectly connected to
Django, since the task could be performed in most web frameworks.
However, the django-dev list is reserved to discussions about the
development *of* Django.

For questions about development *with* Django you should address the
django-users list.

Thank you.

regards
 Steve

Shao wrote:
> Dear ALL,
>
> I am very much interested in using a HTML to fire off a data
> processing script to run over the internet.
>
> I will be very grateful if you can advise me on passing parameters
> from an HTML form to a script and fire the script off to carry out
> full execution, monitor its progress and return an innerHTML with link
> (a name string of) the file generated back to allow user to download
> the result.
>
> The script run over a period of 2 to 3 minutes.
>
>
> Necessary are  the following steps:
> * user submits data to the server from a web page using an HTML form;
> server redirects the browser to a status page that says "script is
> running";
> * in the background, the server fires off a worker script that does
> the job; the server monitors the script to see when it's done (this
> monitoring can be tied to the status page: when the user reloads the
> status page, the server looks at the script to check if it's done);
> * user refreshes the page (or the page refreshes itself) until script
> is finished, and the page says "script finished; here are the results"
> * some kind of cleanup of data, either with a cron job or manually by
> the user
>
> Regards.
>
> Shao
>
> >
>
>   



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread Ludvig Ericson

On Nov 19, 2008, at 12:26, taleinat wrote:
> What about having Django's memcached cache backend implement its own
> hashing algorithm? This could be the "standard" one used in
> libmemcache and cmemcache, or perhaps a consistent hashing algorithm
> [1] such as libketama[2].


As I said in the ticket[1], using python-memcached with
cmemcache_hash[2] would probably be the most beneficial way to go.

So,
1. Try cmemcache
2. Try python-memcached
2.1. Try cmemcache_hash

Also, as said earlier, we should definitely have a setting for
enforcing one or the other.

[1]: http://code.djangoproject.com/ticket/9324
[2]: http://pypi.python.org/pypi/cmemcache_hash

(Apologies if the formatting got weird, but Mail.app is being real lame.)
- Ludvig

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django documentation index redesigned

2008-11-19 Thread Haroldo Stenger
Adrian,

I think this reorganization of Django's documentation is a Big Step Forward.
Congratulations. Not only this will be very important for Django adoption
and public consideration, but also it will model other systems'
documentation, methinks.

best,

haroldo

On Tue, Nov 18, 2008 at 5:44 AM, Adrian Holovaty <[EMAIL PROTECTED]> wrote:

>
> After months of being frustrated (and hearing other people being
> frustrated) with our newish documentation index, I took some time
> tonight to reorganize the links, to make it easier and faster to find
> things. Take a look here:
>
> http://docs.djangoproject.com/en/dev/
>
> I opted for a much more compact approach, with related links
> side-by-side. The previous version tended to organize each link based
> on whether it was reference vs. an overview, but this new version
> organizes by topic, instead.
>
> The "Other batteries included" section is kind of a cop out. It's a
> bit past my bedtime, so that's all I could come up with in a hurry.
>
> Hope people find this easier and faster to use!
>
> Adrian
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Using a HTML to fire off a data processing script on the server (REST or SOAP)

2008-11-19 Thread Shao

Dear ALL,

I am very much interested in using a HTML to fire off a data
processing script to run over the internet.

I will be very grateful if you can advise me on passing parameters
from an HTML form to a script and fire the script off to carry out
full execution, monitor its progress and return an innerHTML with link
(a name string of) the file generated back to allow user to download
the result.

The script run over a period of 2 to 3 minutes.


Necessary are  the following steps:
* user submits data to the server from a web page using an HTML form;
server redirects the browser to a status page that says "script is
running";
* in the background, the server fires off a worker script that does
the job; the server monitors the script to see when it's done (this
monitoring can be tied to the status page: when the user reloads the
status page, the server looks at the script to check if it's done);
* user refreshes the page (or the page refreshes itself) until script
is finished, and the page says "script finished; here are the results"
* some kind of cleanup of data, either with a cron job or manually by
the user

Regards.

Shao

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Proposal: Composite Foreign Keys

2008-11-19 Thread David Cramer
You guys are really bad about keeping on topic. :)

On Wed, Nov 19, 2008 at 9:32 AM, Joey Wilhelm <[EMAIL PROTECTED]> wrote:

> Hanne,
>
> I ran into that exact problem myself and filed the following ticket related
> to the issue: http://code.djangoproject.com/ticket/9519
>
> I also found that you could run your own delete operation through the ORM,
> however, to work around this. It's more work and certainly not in line with
> DRY or any of the normal sensibilities... but it does at least work. Here's
> an example of that, built off the example in the ticket:
> http://dpaste.com/92116/
>
> On Wed, Nov 19, 2008 at 05:07, Hanne Moa <[EMAIL PROTECTED]> wrote:
>>
>> Funny thing though, got a manytomany-table without a primary key: I
>> can add a new row easily enough, the only problem is in delete()-ing
>> specific rows, since that seems to use the primary key. So I clear()
>> and re-add instead of delete()-ing, since that doesn't use the primary
>> key. Fragile, but it beats doing the puppy-eyes to the DBA.
>>
>
>
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Proposal: Composite Foreign Keys

2008-11-19 Thread Joey Wilhelm
Hanne,

I ran into that exact problem myself and filed the following ticket related
to the issue: http://code.djangoproject.com/ticket/9519

I also found that you could run your own delete operation through the ORM,
however, to work around this. It's more work and certainly not in line with
DRY or any of the normal sensibilities... but it does at least work. Here's
an example of that, built off the example in the ticket:
http://dpaste.com/92116/

On Wed, Nov 19, 2008 at 05:07, Hanne Moa <[EMAIL PROTECTED]> wrote:
>
> Funny thing though, got a manytomany-table without a primary key: I
> can add a new row easily enough, the only problem is in delete()-ing
> specific rows, since that seems to use the primary key. So I clear()
> and re-add instead of delete()-ing, since that doesn't use the primary
> key. Fragile, but it beats doing the puppy-eyes to the DBA.
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Remove "old docs" message from Django docs

2008-11-19 Thread Adam Nelson
Actually, this is a typical URL from Google:

 1. Search "django models"
 2. Get
http://docs.djangoproject.com/en/dev/ref/models/instances/?from=olddocs

IMHO, the correct solution is an actual 301 (permanent) redirect to take
Google and others where the "current" URL of the page is.  Unfortunately,
the ?from=olddocs querystring is what Google thinks is the definitive model
reference.  ?from=olddocs needs to still be supported, but if you Google
"Django Models", I don't see any reason to be sent to the 0.96 version of
the page.  The URL should be whatever the current method is.   Part of the
problem is that the Google sitemap hasn't been updated in 2 years and looks
abandoned:

http://code.djangoproject.com/log/djangoproject.com/django_website/sitemaps.py

This is what I think is best but it will take some work and seems lower
priority than actual development:

1. http://docs.djangoproject.com/en/dev/ref/models/instances/?from=olddocsshould
301 (permanent) redirect to
http://docs.djangoproject.com/en/ref/models/instances - it would be nice if
this weren't necessary, but it's too late judging from Google's index.

2. http://docs.djangoproject.com/en/dev/ref/models/instances (with the dev/)
should work, but should show the message telling the user that this is the
development version of this document with a link to
http://docs.djangoproject.com/en/ref/models/instances (without dev/ - the
"current" version of the page which is an alias for whatever the current
version of the codebase is)

3. http://docs.djangoproject.com/en/1.0/ref/models/instances should work
with the 1.0 branch of that document with the same message as #2 giving a
link to the person to http://docs.djangoproject.com/en/ref/models/instances.
The message here would say something like "This is the absolute URL for this
version of this document forever.  Unless necessary, please use the current
documentation at http://docs.djangoproject.com/en/ref/models/instances.

#3 seems redundant but I think it's the best solution to the fact that the
current stable documentation lasts only as long as that release.  As
releases become more common (1.0.1,1.0.2, etc...), this will become more
difficult to manage than it ever has been before.

If somebody would like, I can start a ticket about this although it will
probably take some hashing to figure out the truly correct solution - and
I'm not in the best position to implement this.

On Tue, Nov 18, 2008 at 8:01 PM, Ludvig Ericson <[EMAIL PROTECTED]>wrote:

>
> On Nov 19, 2008, at 00:13, Russell Keith-Magee wrote:
> > That is exactly what _is_ happening. You only get the olddocs
> > parameter if you are visiting from a djangoproject.com/docs URL. If
> > you go in via docs.djangoproject.com, you don't get the olddocs
> > warning.
> >
> > The confusion may be caused by the fact that the Google index (which
> > is used by Django's documentation search) still contains a lot of
> > olddocs pages.
>
> So uh, I agree with the original post a lot. I personally go edit the
> URL all the time so I don't have that big red flashy thing.
>
> The obvious solution -- to me -- is to do a redirect if the referrer
> is not djangoproject.com? To the same URL without the from=olddocs.
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Proposal: Composite Foreign Keys

2008-11-19 Thread Hanne Moa

On Mon, Nov 17, 2008 at 00:55, Frank Liu <[EMAIL PROTECTED]> wrote:
> Plus, if you had a chance to look at the type of databases that I have
> to deal with by day, you will see why this is so important to me.

Yo. Not that big, but there's a DBA involved for some of 'em. Let's
just say I can't use the auth-system or anything depending on that :)

Funny thing though, got a manytomany-table without a primary key: I
can add a new row easily enough, the only problem is in delete()-ing
specific rows, since that seems to use the primary key. So I clear()
and re-add instead of delete()-ing, since that doesn't use the primary
key. Fragile, but it beats doing the puppy-eyes to the DBA.


HM

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django & memcache hashing

2008-11-19 Thread taleinat

On Oct 8, 10:54 am, Malcolm Tredinnick wrote:
> On Wed, 2008-10-08 at 10:37 +0200, Ludvig Ericson wrote:
> > Hello,
> > I had issues with Django opting to use cmemcache and secondarily  
> > python-memcached.
>
> > This behavior is extremely dangerous-- lest you be aware, cmemcache  
> > and python-memcached disagree on how to choose memcached in a list.  
> > Now set up four-five machines with slightly varying needs, disaster.
>
> > The only two possible solutions:
> > 1. ONLY use either cmemcache or python-memcached.
> > 2. Use either cmemcache, or python-memcached with my cmemcache_hash  
> > module which makes python-memcachedhashlike cmemcache does in  
> > choosing server.
>
> Could you create a documentation patch, or at least open a ticket about
> this, please, so that somebody else creates a documentation patch? This
> is definitely worth recording in the docs. It's ultimately a
> configuration issue with the people setting up the four or five machines
> (although see below for how we can make it controllable without harming
> the status quo).
>
> Have you filed your patch for python-memcached with the upstream
> maintainer yet? He's usually fairly responsive to bug reports. I haven't
> had any personal dealings with the cmemcached guys, but they might be
> interested in some sort of common agreement on hashing algorithms as
> well.
>
> Django should continue to try to do both imports by default, since even
> if we introduced a setting, there's still no guarantee that the setting
> would be the same on all the different installations and there's no way
> to ensure that it is automatically (it's a configuration issue). I'm
> very much in favour of "should work out of the box regardless of 
> whichmemcachewrapper you have installed" for the time being, at least.
>
> However, a setting to force it to one or the other (and if no setting is
> present we do the current fallback) would be a good solution for the
> multi-machine installation case. Then somebody scaling to multiple
> machines could use the same settings file with confidence that things
> would fail with prejudice if they configured things correctly but left
> off installing cmemcached.
>
> Best of all (and in parallel), though, would be the idea where the
> different memcached modules used the same hashing algorithm.

What about having Django's memcached cache backend implement its own
hashing algorithm? This could be the "standard" one used in
libmemcache and cmemcache, or perhaps a consistent hashing algorithm
[1] such as libketama[2].

- Tal


[1]http://www.socialtext.net/memcached/index.cgi?
faq#what_is_a_consistent_hashing_client

[2]http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-
_a_consistent_hashing_algo_for_memcache_clients

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---