Re: Django Persistent DB connections - A different approach

[nipun batra]

Thanks.Yes understood the flaw in my approach

On Thu, Dec 15, 2011 at 12:15 AM, Alec Taylor wrote:

> No reason you can't ACID the whole thing
>
> Important info: http://en.wikipedia.org/wiki/Concurrency_control#Methods
>
> On Thu, Dec 15, 2011 at 5:18 AM, akaariai  wrote:
> > A quick warning about your approach: if you use that in multi-threaded
> > environment, you are toasted. You will get two concurrent users for
> > the same connection, and that will not work nicely. For example
> > transaction control will be broken. That is also the reason why that
> > sort of thing is not used in Django.
> >
> > As for connection pooling, there is a patch (ticket 17258) which is
> > somewhat likely to get in that should allow much easier creation of
> > connection pools by external projects.
> >
> >  - Anssi
> >
> > On Dec 14, 1:33 pm, nipunreddevil  wrote:
> >> Hi,
> >>
> >> I have gone similar threads likehttp://
> stackoverflow.com/questions/1125504/django-persistent-database...
> >> and other stuff on same topic. However Django doesn't officially
> >> support persistent connections to MySQL and Mongo(to my limited
> >> knowledge).So i tried avoiding a lot of stuff and tried to make it
> >> simple.So what i did was in my views.py made global connection
> >> variables for both MongoDB and MySQL,something like:
> >>
> >> from pymongo import Connection
> >> import MySQLdb
> >> global
> >> mongo_connection,mongo_db,collection,mysql_connection,mysql_cursor
> >> mysql_connection = MySQLdb.connect (host = "localhost",
> >>user = "root",
> >>passwd = "password",
> >>db = "demo")
> >> mysql_cursor = mysql_connection.cursor ()
> >> mongo_connection = Connection()
> >> mongo_db = mongo_connection.test_database
> >> collection = mongo_db.test_collection
> >> So after this when the required view is called as per URL requested,i
> >> dump the data in the two databases.Like:
> >>
> >> mysql_cursor.execute('''INSERT INTO
> >> table_name(l,n_n,n_id,s_n,s_id,u,r) VALUES
> >> (%s,%s,%s,%s,%s,%s,%s)''',
> >> (l,n_n,n_id,s_name,s_id,u,re)
> >> )
> >> And similarly i did for saving to MongoDB.
> >>
> >> Obviously there's this flaw in this approach that i am not closing the
> >> connection anywhere.But this approach does seem to work and work well.
> >>
> >> Why is this sort of approach not used?
> >>
> >> How can i measure the performance improvements i get by using this
> >> approach v/s letting Django create a new connection to DB on each
> >> call.
> >>
> >> Also a batch insert is supposed to make things even better,by reducing
> >> calls to DB.How can such a concept be implemented within view
> >> definition?
> >>
> >> Here is how my application behaved before i had used my method of
> >> trying to make a persistent connection and had let Django take care of
> >> it
> >>
> >> mysql> show status like '%onn%';
> >> +--++
> >> | Variable_name| Value  |
> >> +--++
> >> | Aborted_connects | 0  |
> >> | Connections  | 164359 |
> >> | Max_used_connections | 3  |
> >> | Ssl_client_connects  | 0  |
> >> | Ssl_connect_renegotiates | 0  |
> >> | Ssl_finished_connects| 0  |
> >> | Threads_connected| 1  |
> >> +--++
> >> 7 rows in set (0.00 sec)
> >> After a few seconds when i ran the same query i got:
> >>
> >> mysql> show status like '%onn%';
> >> +--++
> >> | Variable_name| Value  |
> >> +--++
> >> | Aborted_connects | 0  |
> >> | Connections  | 175047 |
> >> | Max_used_connections | 3  |
> >> | Ssl_client_connects  | 0  |
> >> | Ssl_connect_renegotiates | 0  |
> >> | Ssl_finished_connects| 0  |
> >> | Threads_connected| 1  |
> >> +--++
> >> 7 rows in set (0.00 sec)
> >> However after using my approach the number of connections didn't
> >> increase.
> >>
> >> NB:I have posted same post onhttp://
> stackoverflow.com/questions/8502814/django-persistent-db-conne...
> >> but thought this might be a good place to get the answer
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> 

Re: Django Persistent DB connections - A different approach

[Alec Taylor]

No reason you can't ACID the whole thing

Important info: http://en.wikipedia.org/wiki/Concurrency_control#Methods

On Thu, Dec 15, 2011 at 5:18 AM, akaariai  wrote:
> A quick warning about your approach: if you use that in multi-threaded
> environment, you are toasted. You will get two concurrent users for
> the same connection, and that will not work nicely. For example
> transaction control will be broken. That is also the reason why that
> sort of thing is not used in Django.
>
> As for connection pooling, there is a patch (ticket 17258) which is
> somewhat likely to get in that should allow much easier creation of
> connection pools by external projects.
>
>  - Anssi
>
> On Dec 14, 1:33 pm, nipunreddevil  wrote:
>> Hi,
>>
>> I have gone similar threads 
>> likehttp://stackoverflow.com/questions/1125504/django-persistent-database...
>> and other stuff on same topic. However Django doesn't officially
>> support persistent connections to MySQL and Mongo(to my limited
>> knowledge).So i tried avoiding a lot of stuff and tried to make it
>> simple.So what i did was in my views.py made global connection
>> variables for both MongoDB and MySQL,something like:
>>
>> from pymongo import Connection
>> import MySQLdb
>> global
>> mongo_connection,mongo_db,collection,mysql_connection,mysql_cursor
>> mysql_connection = MySQLdb.connect (host = "localhost",
>>                        user = "root",
>>                        passwd = "password",
>>                        db = "demo")
>> mysql_cursor = mysql_connection.cursor ()
>> mongo_connection = Connection()
>> mongo_db = mongo_connection.test_database
>> collection = mongo_db.test_collection
>> So after this when the required view is called as per URL requested,i
>> dump the data in the two databases.Like:
>>
>> mysql_cursor.execute('''INSERT INTO
>> table_name(l,n_n,n_id,s_n,s_id,u,r) VALUES
>> (%s,%s,%s,%s,%s,%s,%s)''',
>> (l,n_n,n_id,s_name,s_id,u,re)
>> )
>> And similarly i did for saving to MongoDB.
>>
>> Obviously there's this flaw in this approach that i am not closing the
>> connection anywhere.But this approach does seem to work and work well.
>>
>> Why is this sort of approach not used?
>>
>> How can i measure the performance improvements i get by using this
>> approach v/s letting Django create a new connection to DB on each
>> call.
>>
>> Also a batch insert is supposed to make things even better,by reducing
>> calls to DB.How can such a concept be implemented within view
>> definition?
>>
>> Here is how my application behaved before i had used my method of
>> trying to make a persistent connection and had let Django take care of
>> it
>>
>> mysql> show status like '%onn%';
>> +--++
>> | Variable_name            | Value  |
>> +--++
>> | Aborted_connects         | 0      |
>> | Connections              | 164359 |
>> | Max_used_connections     | 3      |
>> | Ssl_client_connects      | 0      |
>> | Ssl_connect_renegotiates | 0      |
>> | Ssl_finished_connects    | 0      |
>> | Threads_connected        | 1      |
>> +--++
>> 7 rows in set (0.00 sec)
>> After a few seconds when i ran the same query i got:
>>
>> mysql> show status like '%onn%';
>> +--++
>> | Variable_name            | Value  |
>> +--++
>> | Aborted_connects         | 0      |
>> | Connections              | 175047 |
>> | Max_used_connections     | 3      |
>> | Ssl_client_connects      | 0      |
>> | Ssl_connect_renegotiates | 0      |
>> | Ssl_finished_connects    | 0      |
>> | Threads_connected        | 1      |
>> +--++
>> 7 rows in set (0.00 sec)
>> However after using my approach the number of connections didn't
>> increase.
>>
>> NB:I have posted same post 
>> onhttp://stackoverflow.com/questions/8502814/django-persistent-db-conne...
>> but thought this might be a good place to get the answer
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Django Persistent DB connections - A different approach

[akaariai]

A quick warning about your approach: if you use that in multi-threaded
environment, you are toasted. You will get two concurrent users for
the same connection, and that will not work nicely. For example
transaction control will be broken. That is also the reason why that
sort of thing is not used in Django.

As for connection pooling, there is a patch (ticket 17258) which is
somewhat likely to get in that should allow much easier creation of
connection pools by external projects.

 - Anssi

On Dec 14, 1:33 pm, nipunreddevil  wrote:
> Hi,
>
> I have gone similar threads 
> likehttp://stackoverflow.com/questions/1125504/django-persistent-database...
> and other stuff on same topic. However Django doesn't officially
> support persistent connections to MySQL and Mongo(to my limited
> knowledge).So i tried avoiding a lot of stuff and tried to make it
> simple.So what i did was in my views.py made global connection
> variables for both MongoDB and MySQL,something like:
>
> from pymongo import Connection
> import MySQLdb
> global
> mongo_connection,mongo_db,collection,mysql_connection,mysql_cursor
> mysql_connection = MySQLdb.connect (host = "localhost",
>                        user = "root",
>                        passwd = "password",
>                        db = "demo")
> mysql_cursor = mysql_connection.cursor ()
> mongo_connection = Connection()
> mongo_db = mongo_connection.test_database
> collection = mongo_db.test_collection
> So after this when the required view is called as per URL requested,i
> dump the data in the two databases.Like:
>
> mysql_cursor.execute('''INSERT INTO
> table_name(l,n_n,n_id,s_n,s_id,u,r) VALUES
> (%s,%s,%s,%s,%s,%s,%s)''',
> (l,n_n,n_id,s_name,s_id,u,re)
> )
> And similarly i did for saving to MongoDB.
>
> Obviously there's this flaw in this approach that i am not closing the
> connection anywhere.But this approach does seem to work and work well.
>
> Why is this sort of approach not used?
>
> How can i measure the performance improvements i get by using this
> approach v/s letting Django create a new connection to DB on each
> call.
>
> Also a batch insert is supposed to make things even better,by reducing
> calls to DB.How can such a concept be implemented within view
> definition?
>
> Here is how my application behaved before i had used my method of
> trying to make a persistent connection and had let Django take care of
> it
>
> mysql> show status like '%onn%';
> +--++
> | Variable_name            | Value  |
> +--++
> | Aborted_connects         | 0      |
> | Connections              | 164359 |
> | Max_used_connections     | 3      |
> | Ssl_client_connects      | 0      |
> | Ssl_connect_renegotiates | 0      |
> | Ssl_finished_connects    | 0      |
> | Threads_connected        | 1      |
> +--++
> 7 rows in set (0.00 sec)
> After a few seconds when i ran the same query i got:
>
> mysql> show status like '%onn%';
> +--++
> | Variable_name            | Value  |
> +--++
> | Aborted_connects         | 0      |
> | Connections              | 175047 |
> | Max_used_connections     | 3      |
> | Ssl_client_connects      | 0      |
> | Ssl_connect_renegotiates | 0      |
> | Ssl_finished_connects    | 0      |
> | Threads_connected        | 1      |
> +--++
> 7 rows in set (0.00 sec)
> However after using my approach the number of connections didn't
> increase.
>
> NB:I have posted same post 
> onhttp://stackoverflow.com/questions/8502814/django-persistent-db-conne...
> but thought this might be a good place to get the answer

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Django Persistent DB connections - A different approach

[nipunreddevil]

Hi,

I have gone similar threads like
http://stackoverflow.com/questions/1125504/django-persistent-database-connection
and other stuff on same topic. However Django doesn't officially
support persistent connections to MySQL and Mongo(to my limited
knowledge).So i tried avoiding a lot of stuff and tried to make it
simple.So what i did was in my views.py made global connection
variables for both MongoDB and MySQL,something like:

from pymongo import Connection
import MySQLdb
global
mongo_connection,mongo_db,collection,mysql_connection,mysql_cursor
mysql_connection = MySQLdb.connect (host = "localhost",
   user = "root",
   passwd = "password",
   db = "demo")
mysql_cursor = mysql_connection.cursor ()
mongo_connection = Connection()
mongo_db = mongo_connection.test_database
collection = mongo_db.test_collection
So after this when the required view is called as per URL requested,i
dump the data in the two databases.Like:

mysql_cursor.execute('''INSERT INTO
table_name(l,n_n,n_id,s_n,s_id,u,r) VALUES
(%s,%s,%s,%s,%s,%s,%s)''',
(l,n_n,n_id,s_name,s_id,u,re)
)
And similarly i did for saving to MongoDB.

Obviously there's this flaw in this approach that i am not closing the
connection anywhere.But this approach does seem to work and work well.

Why is this sort of approach not used?

How can i measure the performance improvements i get by using this
approach v/s letting Django create a new connection to DB on each
call.

Also a batch insert is supposed to make things even better,by reducing
calls to DB.How can such a concept be implemented within view
definition?

Here is how my application behaved before i had used my method of
trying to make a persistent connection and had let Django take care of
it

mysql> show status like '%onn%';
+--++
| Variable_name| Value  |
+--++
| Aborted_connects | 0  |
| Connections  | 164359 |
| Max_used_connections | 3  |
| Ssl_client_connects  | 0  |
| Ssl_connect_renegotiates | 0  |
| Ssl_finished_connects| 0  |
| Threads_connected| 1  |
+--++
7 rows in set (0.00 sec)
After a few seconds when i ran the same query i got:

mysql> show status like '%onn%';
+--++
| Variable_name| Value  |
+--++
| Aborted_connects | 0  |
| Connections  | 175047 |
| Max_used_connections | 3  |
| Ssl_client_connects  | 0  |
| Ssl_connect_renegotiates | 0  |
| Ssl_finished_connects| 0  |
| Threads_connected| 1  |
+--++
7 rows in set (0.00 sec)
However after using my approach the number of connections didn't
increase.

NB:I have posted same post on
http://stackoverflow.com/questions/8502814/django-persistent-db-connections-a-different-approach
but thought this might be a good place to get the answer

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Different approach?

[google torp]

Hi.

As I'm not sure exactly what your goals with this site is, I'll be
guessing a
bit here. Using the admin to create customers ect, can be a solution,
but
having access to the admin also means having access to a whole bunch
of
stuff that you can mess with. You could instead make some views that
will
display some modelforms that you can use to create what you need.
Maybe
you also some some views that will display customers based per city
postal code, city areas or whatever suits best. You could also add
order
tracking, so you can see what pizza's are selling most and see if
repeat
customers are buying the same (kind of) pizzas. All of these would
also
require views. There are lots of possibilities to extend. Also using
the admin
as the main part of the site should only be done, if the site is only
used by
very few trusted people.

~Jakob

On 15 Apr., 05:40, Mariano Sokal  wrote:
> Well, i'm also new :)
>
> I've been using django for a couple of days trying to learn while developing
> a Pizza Delivery Software... (yes, I got bored doing blogs) and since it is
> not a traditional website where one has an admin area and a
> regular-user-live-site area, I feel that almost every functionality that I
> want for my app can be accomplished just by using admin.
>
> Ok, as I told you, I am new... but I need to add customers, add addresses,
> add orders... and after doing all the models and registering the admin
> models I feel that I'm almost done with the application. Is that the way you
> do an application where there's no public website but is more like a
> point-of-sale?
>
> Regards,
> Mariano
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Different approach?

[Mariano Sokal]

Well, i'm also new :)

I've been using django for a couple of days trying to learn while developing
a Pizza Delivery Software... (yes, I got bored doing blogs) and since it is
not a traditional website where one has an admin area and a
regular-user-live-site area, I feel that almost every functionality that I
want for my app can be accomplished just by using admin.

Ok, as I told you, I am new... but I need to add customers, add addresses,
add orders... and after doing all the models and registering the admin
models I feel that I'm almost done with the application. Is that the way you
do an application where there's no public website but is more like a
point-of-sale?

Regards,
Mariano

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: extending user model, a different approach

[Amit Upadhyay]

On 7/26/07, James Bennett <[EMAIL PROTECTED]> wrote:
>
> On 7/25/07, Amit Upadhyay <[EMAIL PROTECTED]> wrote:
> > I realized a new way to extend user models, it is simpler than official
> > get_profile approach of django[1] as there is only one model to work
> with,
> > and relationships are kept on the right model, and gives a less hacky
> feel
> > than "replaces_module" method[2].
>
> Personally, I'd disagree about it feeling "hacky"; it's using internal
> metasystem methods to pry open the model class and shove things into
> it, which is, well, hacky ;)


:-)

In this solution, however, every site shoves all its customization
> straight into the User model that's shared among them all, which
> raises a lot of ugly questions about what happens when those
> customizations collide with one another. Keeping per-site
> customization in a manageable, separate namespace is a good thing ;)


Well, AUTH_PROFILE_MODULE is not going away, it can be used in multi site
cases, but for simpler cases, this approach is much simpler. Also this can
be used to extend any django.contrib model, you can add "favicon_url" in
Site, or "description" in auth.Group for example.

-- 
Amit Upadhyay
http://www.amitu.com/blog/
+91-9820-295-512

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: extending user model, a different approach

[James Bennett]

On 7/25/07, Amit Upadhyay <[EMAIL PROTECTED]> wrote:
> I realized a new way to extend user models, it is simpler than official
> get_profile approach of django[1] as there is only one model to work with,
> and relationships are kept on the right model, and gives a less hacky feel
> than "replaces_module" method[2].

Personally, I'd disagree about it feeling "hacky"; it's using internal
metasystem methods to pry open the model class and shove things into
it, which is, well, hacky ;)

Also, it's basically unusable on a multi-site installation. With
AUTH_PROFILE_MODULE each site has its own settings file and can
specify a different profile module, which means it's not possible for
the different sites to interfere with each other's customizations.

In this solution, however, every site shoves all its customization
straight into the User model that's shared among them all, which
raises a lot of ugly questions about what happens when those
customizations collide with one another. Keeping per-site
customization in a manageable, separate namespace is a good thing ;)


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

extending user model, a different approach

[Amit Upadhyay]

Hi,

I realized a new way to extend user models, it is simpler than official
get_profile approach of django[1] as there is only one model to work with,
and relationships are kept on the right model, and gives a less hacky feel
than "replaces_module" method[2].

User.add_to_class("openid", models.URLField(blank=True))
User._meta.admin.fields += (
("AmitCom Extensions", { 'fields': ('openid', ) }),
)

*class* UserExtension(object):
*def* _get_comments(self):
*return* Comment.objects.get(user=self, is_public=True)
public_comments = property(_get_comments)
*del* _get_comments

User.__bases__ = User.__bases__ + ( UserExtension, )

Read my blog post about the same[3] for a longer discussion about pros and
cons. How does it sound?

1. http://www.b-list.org/weblog/2006/06/06/django-tips-extending-user-model
2. http://code.djangoproject.com/wiki/ExtendedUserModel
3. http://www.amitu.com/blog/2007/july/django-extending-user-model/

-- 
Amit Upadhyay
http://www.amitu.com/blog/
+91-9820-295-512

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---