Re: Feedback requested: One last time with multi-db
On Jan 23, 7:05 am, Russell Keith-Magee wrote: > A full explanation, including an example is in the documentation > portion of the patch, attached to ticket #12542. > > Comments? This fixed our only (known) blocking problem towards 1.2, thanks so much. Brett -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
On Jan 7, 7:33 pm, Russell Keith-Magee wrote:
> then my understanding of your proposal is that the only change is that
> read-slave won't get created under the test setup. But doesn't that
> mean that::
>
> MyModel.objects.using('read-slave').filter(...)
>
> will fall over?
No, not in my mental image of the setup. Take the following,
DEFAULT_ENGINE = 'postgresql_psycopg2'
DEFAULT_NAME = 'my_database'
DATABASES = {
'default': {
'ENGINE': DEFAULT_ENGINE,
'NAME': DEFAULT_NAME,
},
'read_slave': {
'ENGINE': DEFAULT_ENGINE,
'NAME': DEFAULT_NAME,
'TEST_NAME': None,
},
}
So the important thing here is that 'read_slave' *is* defined, but in
my local test settings it uses the same `DATABASE_NAME' (and host,
user, password, port) as my `default'. The `TEST_NAME = None' change
will simply allow me to get past the error caused when `read_slave'
tries to drop and create a database that `default' has an open session
to (it just dropped and created itself, after all).
Now in code like,
MyModel.objects.using('read-slave').filter(...)
That should be a valid connection (`DATABASES['read_slave']') but it's
actually connecting to the exact same DB as `default', so a filter
should find objects created on `default', just like you'd imagine in a
real world read-slave setup.
Does that make more sense? There's really nothing magic going on
here, it's only a matter of telling it not to drop/create the DB. I
think maybe `TEST_NAME = None' could be confusing? I didn't mean to
imply that the alias wasn't properly setup and functional.
Regards,
Brett
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
On Jan 5, 8:09 pm, Russell Keith-Magee wrote:
> If you're actually doing master/slave in the wild, your guidance may
> actually be more enlightening than my theoretical navel gazing. In
> particular - how have you got master/slave configured? How do you find
> and select slave databases? How does that approach degrade when
> DATABASES suddenly has less entries (including the case of a config
> with no slaves)?
Yes, we're actually doing read-slave queries on Django 1.0.x using
some private API hacks.
We basically have the same layout of DATABASES that multidb went with,
but we only use different managers to dispatch queries. In other
words, `Foo.rs_objects.all()' vs `Foo.objects.all()'. It's pretty
basic, but it's worked for us.
So that's equivalent to the `using' syntax, you can just imagine we
have places in our code where the developer knows that read-slave
replication isn't a problem and we want to offload a query, so
`Foo.objects.using('read_slave')...' is used. We don't do any special
selection right now, `DATABASES['read_slave']' is hard coded per
deployment instance, different instances might use different read-
slaves for various reasons but those reasons also require us to use
whole different app servers too, and so those requests are chosen by a
frontend proxy rather than some in-app magic.
Anyway, most of that doesn't really matter, I think. What matters is
that we don't do any special degrading if `DATABASES' is different.
As soon as you use `using' (or our equivalent) you're hard coding the
use of another DB name, so in development we just have `DATABASES
['read_slave']' use the same settings as `default' does.
So in the end the `TEST_NAME=None' solution works well for our case at
least, I would imagine for any number of read-slaves you'd want to be
able to point them at the `default' DB (without doing a dump and sync)
during tests - I mean, that's what a read-slave is, no?
Regards,
Brett
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
On Dec 22 2009, 4:27 pm, Russell Keith-Magee wrote: > I'll need to cogitate on this over my Christmas pudding :-) Did you come to any conclusions, or need any more feedback on the read- slave testing issue? I in no way mean to rush, I just wanted to make sure I didn't (and don't) miss anything as I'm more than happy to test any changes against our rather large and quirky setup. Thanks, Brett -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
On Dec 22, 4:27 pm, Russell Keith-Magee
wrote:
> * Allow TEST_NAME=None to mean "don't try and instantiate this
> database in test mode"
That sounds good, too.
> * Allow a top level TEST_DATABASES setting; TEST_DATABASES would
> override DATABASES; if TEST_DATABASES isn't defined, then TEST_NAME
> would be used.
This would have to function differently than the current DATABASES
setting, though, right? Otherwise I don't see how the same problem
would be avoided ('default' and 'read_slave' both point the same
physical DB).
> I'll need to cogitate on this over my Christmas pudding :-) Any other
> suggestions welcome.
Awesome, thanks much.
Brett
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
On Dec 19, 6:48 am, Russell Keith-Magee wrote: > You're right - read slaves are an intended common use case I know the branch landed but I'd like to mention another issue regarding read-slaves, hope that's OK. :) Running tests against code that uses master and read-slaves (but actually point at the same exact DB while testing) is currently not possible. When the tests begin each DB is expected to be started fresh (the runner stops if it cannot), so you can't use the same DB name/ host for two entries in settings.DATABASES. I think it's awfully common for people not develop with *actual* read-slaves on their local machine, but rather to use the abstractions available and point their "read-slave" at the same information as their default DB. I could just set TEST_NAME for the read-slave to something else, but in our production code and tests (as an example) we'll have some test inserting data on master and another using it via a read-slave. Unless I'm missing something, we'd have to create all data on both the master and read-slaves in order to test properly, which is awfully awkward and possibly confusing. Any thoughts? Off the top of my head all I can think of is that the test setup could check if any DBs match up in name/host and not try to drop/create after the first. Thanks, Brett -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
On Dec 18, 9:50 am, Alex Gaynor wrote: > I'm wondering if perhaps the most prudent thing to do would be to > simply remove this check. The end result will be you'll get an > integrity error on Postgres/Oracle when you try to save (and SQLite > and MySQL will just let you do whatever). That would certainly work for us. I guess it's really a question of how it'd affect users with multiple truly distinct databases. I think it's a nice error for people "new" to Django and the concept of multiple databases, but when you're using more than one DB already... Brett -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Final call for feedback: Multi-db
I'm not sure if 1.2 intended to fully support read-slaves, but I'll
post this quick anyway as we've just run into it while trying to
upgrade at DISQUS.
You might think that having support for multiple databases implies
that using a read-slave would Just Work, and that's mostly true.
There's one edge case I've run into when you try to relate objects
using instances created from different mirrors of the same database.
Because of the checks against instance._state.db you can't select an
object from a read-slave and assign it to a foreign key relation (and
probably other relations) on the master, even though you know this is
a mirror and not a different database.
Here's a quick code example: http://dpaste.org/Bozd/
The only solution I've thought of (and I haven't thought long, I just
ran into this) is another database setting where you could tell Django
that this DB is a mirror of another (by name?) so that
instance._state.db on a read-slave created object actually holds the
value of DATABASES['that_read_slave']['mirror_of'] (or whatever key).
In other words a User selected from 'read_slave' might actually have a
user_instance._state.db value of 'default', because that's it's true
home, and any relations should be compared to that.
I would think read-slaves would be a pretty common application of
multidb, but I can only speak to our use case. I know it's a bit late
in the game, but we'll have to work up our own local fix or go with a
proper one before we can deploy 1.2. And to think I was so happy
about how many local Django patches I was able to remove going from
1.0->1.2. ;)
Amazing work, Alex & Russell, many thanks.
Regards,
Brett
On Dec 17, 11:43 pm, Russell Keith-Magee
wrote:
> Hi all,
>
> This is a second and final call for feedback on the multidb branch.
>
> Barring any objections or the discovery of major problems, my
> intention is to commit this early next week, hitting the alpha 1
> feature deadline by the skin of our collective teeth :-)
>
> There has been one big change since the last call for feedback -
> thanks to Justin Bronn, GIS is now fully multi-db compliant.
>
> There have also been a couple of small changes - mostly integrating
> the contrib applications with multidb features. For example, the
> contenttypes app now maintains a cache of content type objects that is
> multi-db aware.
>
> The only really visible change is a new 'db_manager()' operator on
> Managers. This is used to obtain a Manager instance that is bound to a
> specific database. This is required because:
>
> Author.objects.using('foo')
>
> will return a QuerySet that is bound to foo - however, methods like
> User.objects.create_user(...) and
> Permission.objects.get_by_natural_key(...) are on the Manager, not the
> QuerySet.
>
> So, you can now call:
>
> Author.objects.db_manager('foo')
>
> which will return a Manager (Author's default manager) that is bound
> to the foo databse. Subsequent calls to using() can change this
> binding if necessary.
>
> At the last call for feedback, questions were raised about admin
> support. I've done some investigation, and it turns out that you can
> write ModelAdmin definitions that bind a model to a different
> database. I'm in the process of documenting the exact steps that are
> required. Coming up with a pretty integration layer with admin will be
> left for Django 1.3, when we will be addressing the issue of how to
> provide a good public interface to multidb for common use cases
> (master/slave, sharding, etc)
>
> As always, the code is available in the multi-db SVN branch:
>
> http://code.djangoproject.com/svn/django/branches/soc2009/multidb/
>
> or from Alex's github branch:
>
> http://github.com/alex/django/tree/multiple-db
>
> Again, any and all feedback welcome.
>
> Yours,
> Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Dallas 1.1 sprint - dates?
On Fri, Apr 3, 2009 at 12:38 PM, Nizam Sayeed wrote: > Count me in and a friend of mine as well. > > On Apr 3, 12:33 pm, Preston Timmons wrote: >> I would be interested in coming as well. I assume you guys are OK with either weekend, then? Brett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: I want a pony: Django Cheeseshop
On Wed, Sep 10, 2008 at 11:20 AM, mrts <[EMAIL PROTECTED]> wrote: > And it doesn't handle project-local installation (arguably this > can be done with virtualenv, but that will just clutter user-specific > "app-space" instead of the global one). At some point the Django app you're trying to installed has to go somewhere. If it doesn't go in global or user's local, where do you want it to go? With virtualenv you can (and I would encourage that you) have an env for each Django project, so any installing you do is only 'cluttering' that project, and at that point it isn't even 'clutter'. Brett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: New milestones for ticket triagers
On Sat, Sep 6, 2008 at 12:46 PM, daonb <[EMAIL PROTECTED]> wrote: > So we want a roadmap, to better help with ticket triage. You can reply > with -1 or 0 if you don't like the idea I very much doubt anyone is against a roadmap. Again, Djangocon _just_ started. Let me quote Jacob from this very thread, We'll be using the opportunity of DjangoCon to think out our workflow going forward, and then we'll bring whatever we come up with here for discussion. > don't tell us you're too busy doing grownup things. > We are grownups too and deserve to be treated as such. Again, I don't really know what else to say. Brett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: New milestones for ticket triagers
On Sat, Sep 6, 2008 at 11:41 AM, daonb <[EMAIL PROTECTED]> wrote: > Jacob, please release a roadmap ASAP or let us know what version we > should use for new tickets. It's important to have a roadmap (and I'm > +1 for grasshoper's suggestion) Er, Django 1.0 was only released _3 days_ ago. You know people are literally sitting down to the start of Djangocon right now, right? I don't really know what else to say. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: test coverage [was: web typography support]
On 8/6/07, Emanuele Pucciarelli <[EMAIL PROTECTED]> wrote: > There's Ned Batchelder's coverage.py: http://nedbatchelder.com/code/ > modules/coverage.html . Haven't tried it myself, but I have to > suppose that it's Django-friendly. :) http://siddhi.blogspot.com/2007/04/code-coverage-for-your-django-code.html http://eddymulyono.livejournal.com/62101.html Brett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re[2]: usability issues
On 8/5/07, James Bennett <[EMAIL PROTECTED]> wrote: > Django handles both just fine, returns the same page for both. Not on our live site. And I just made a fresh django-trunk project to test it, didn't work there either. Although aren't resolvers just normal Python REs, so you could possibly have set this for yourself somehow? Regardless, I agree with James in that 'normal users' don't type URLs, and I like having objects available from one exact URL. I'm sure Google is smart, but who knows what search engines do when they see the same things available from different-cased URLs. Sort of like the append_slash problem, right? We don't want to confuse the computer in this case. Brett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
