Re: Database router with read replica

[Antonis Christofides]

Hi,

When I once faced this problem I used django-multidb-router
. It uses the term "pinning"
for the fact that after a query to the master database it "pins" itself to the
master database, i.e. uses the master database exclusively for the rest of the
request. IIRC it works as you say, with a thread-local variable. It wasn't doing
everything I wanted, and merging my pull requests didn't go fast enough, but I
maintained my own fork which worked fine for the project I was working on.

Even if you decide not to use it, it will give you ideas on how to implement the
things you need.

Regards,

Antonis

Antonis Christofides
http://djangodeployment.com


On 2017-08-30 22:16, Matt Pegler wrote:
> I'm hoping for some advice on how to utilize read replica databases with
> Django.  I searched the django-users archive but didn't find much discussion
> about this.  I also can't find many blog posts or discussion elsewhere.  Do
> people have experience or general advice on using read replicas with Django?
>
> I am hoping to implement a database router that will intelligently use a read
> replica while maintaining consistency despite replica lag.  I have a couple
> things I'd like another opinion on:
>
> First is handling replication lag.  Our read replica typically lags ~20ms
> behind master, so we want to route all queries that occur after a write within
> the same request to the master database.  I believe this can be accomplished
> using a thread local in the database router that is reset after each request
> using a middleware.  Does that seem like a reasonable approach?
>
> Second is handling atomic blocks.  As far as I can tell, Django will not
> automatically route all queries within an atomic block to the same database,
> so this needs to be handled within the router.  Is connection.in_atomic_block
> 
>  a
> public API, or is there a better way to tell if we're within an atomic block?
>
> Thanks,
> Matt
> -- 
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CA%2BSd1We3Yq37fYsZu1rVJcxizty6NNG4q-EeVp83wT7aeuh8pQ%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/0d1682c8-42d8-bd62-2be1-1e74d27eef3b%40djangodeployment.com.
For more options, visit https://groups.google.com/d/optout.

Database router with read replica

[Matt Pegler]

I'm hoping for some advice on how to utilize read replica databases with
Django.  I searched the django-users archive but didn't find much
discussion about this.  I also can't find many blog posts or discussion
elsewhere.  Do people have experience or general advice on using read
replicas with Django?

I am hoping to implement a database router that will intelligently use a
read replica while maintaining consistency despite replica lag.  I have a
couple things I'd like another opinion on:

First is handling replication lag.  Our read replica typically lags ~20ms
behind master, so we want to route all queries that occur after a write
within the same request to the master database.  I believe this can be
accomplished using a thread local in the database router that is reset
after each request using a middleware.  Does that seem like a reasonable
approach?

Second is handling atomic blocks.  As far as I can tell, Django will not
automatically route all queries within an atomic block to the same
database, so this needs to be handled within the router.  Is
connection.in_atomic_block

a
public API, or is there a better way to tell if we're within an atomic
block?

Thanks,
Matt

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CA%2BSd1We3Yq37fYsZu1rVJcxizty6NNG4q-EeVp83wT7aeuh8pQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.