Re: Single DB connection during class's lifetime. Metaclass,singleton and __new__() examples and references.
On 12Oct2018 13:28, Ryan Johnson wrote: Thanks for the clarification. If I am creating a class variable, are you suggesting I perform the “if it exists, great, otherwise make it” logic in the __init__ block or in the class definition block? Will that even run in a class definition? The class definition code runs when the class is defined (as Python reads it in your code). The __init__ block runs once each time a new instance of the class is initialised. When do you _want_ this logic to run? That dictates where the logic goes. If you run this in the class definition code it pretty much will unconditionally make a db connection. But in reality (a) you usually want to defer making the connection until you need it to reduce resource usage and (b) you often don't know enough to make the connection at class definition time i.e. you don't know the database host, the credentials, etc - they are often supplied to the initialiser (directly or via some config file). I never see examples do anything besides assignment operations and flow control, although it would follow that if the block allows creation of strings (an object), it would allow creation of connection objects. On the other hand, the __init__ block seems like a natural place to put the cursor instantiation. You can do anything that is sensible in the __init__ block - it is just code. Your goal is to decide what is sensible. Normally the initialiser mosts sets up various arrtributes to sane initial values. It can do complex things, but is usually relatively basic. I'd note, as Thomas did, that the cursor is a control object associated with a query. You can have multiple cursors on a connection, and you often make one from a query, process the query results, then discard the cursor. SO you routinely use several during the lifetime of a connection. Therefore you don't make cursors when you set up the connection; you make them in association with queries. From this answer ( https://stackoverflow.com/questions/25577578/python-access-class-variable-from-instance/25577642#25577642 ) the pythonic way to access the class variable is using type(self), but doesn’t this access the local class’s variable, if someone subclasses my class? Can I specify my class variable via the class name itself? Example: instancevar = ClassName.classvar . I am hoping that because of the way python points labels to objects, this should give my instance an instance var that refers to the class var. Am I right? _Usually_ I access class attributes (which you're calling variables, I believe - they're not) via the instance: def foo(self, blah=None): if blah is None: blah = foo.DEFAULT_BLAH_VALUE ... work with blah ... As you suggest, this will find DEFAULT_BLAH_VALUE from the subclass before it finds it from the superclass. Usually that is what I want - the purpose of subclassing is to (possibly) override the aspects of the superclass. However, you _can_ always reach directly to a specific class to get a value: blah = MySuperCLass.DEFAULT_BLAH_VALUE if that is sensible. All you're doing is changing the way in which the name "DEFAULT_BLAH_VALUE" is found: do I use the instance's name lookup or go somewhere direct? In your case with a persistent database connection associated with a class it would be best to make the "get a connection" logic a class method because the "is there a connection" attribute is associated with the class, not the instance. Methods are, by default, "instance" methods: they are defined like this: def method(self, ...): and you largely work through "self", being the current instance. That is its "context". Class method are defined like this: @classmethod def method(cls, ...) and instead of having an instance as context (with the conventional name 'self"), you have the class (with the conventional name "cls"). These are for methods which _do_ _not_ care about the instance, just the class. So in the case of your database connection, made on demand once per class, you might go: @classmethod def conn(cls): c = cls.connection if c is None: c = connect_to_the_db(.) cls.connection = c return c See that there's no "self" in here? Then your instance methods can look like this: def lookup(self, ): conn = self.conn() cursor = conn.select() ... use the cursor to process the result ... The instance finds the "conn" class method in the usual way: look in the instance, then look in the class hierarchy. This in itself is an argument against making the connection in the __init__ block. Does this help? Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
RE: Single DB connection during class's lifetime. Metaclass,singleton and __new__() examples and references.
Thanks for the clarification.
If I am creating a class variable, are you suggesting I perform the “if it
exists, great, otherwise make it” logic in the __init__ block or in the class
definition block? Will that even run in a class definition? I never see
examples do anything besides assignment operations and flow control, although
it would follow that if the block allows creation of strings (an object), it
would allow creation of connection objects. On the other hand, the __init__
block seems like a natural place to put the cursor instantiation.
From this answer (
https://stackoverflow.com/questions/25577578/python-access-class-variable-from-instance/25577642#25577642
) the pythonic way to access the class variable is using type(self), but
doesn’t this access the local class’s variable, if someone subclasses my class?
Can I specify my class variable via the class name itself? Example: instancevar
= ClassName.classvar .
I am hoping that because of the way python points labels to objects, this
should give my instance an instance var that refers to the class var. Am I
right?
RJ
Sent from Mail for Windows 10
From: Thomas Jollans
Sent: Friday, October 12, 2018 2:05 AM
To: python-list@python.org
Subject: Re: Single DB connection during class's lifetime. Metaclass,singleton
and __new__() examples and references.
On 12/10/2018 01:19, Ryan Johnson wrote:
> I am working on using mysql.connector in a class and have found an example of
> how to create a single connection that spans the lifetime of all instances of
> the class:
>
> https://softwareengineering.stackexchange.com/a/358061/317228
>
> however, I do not understand a few things about the class, including
>
> 1. Why it is subclassed as an object: `class Postgres(object):` ? I thought
> classes were necessarily objects.
This was sometimes necessary in Python 2.
> 2. Why is this portion of code directly addressing the class, instead of
> using the `cls` reference variable?
> connection = Postgres._instance.connection = psycopg2.connect(**db_config)
> cursor = Postgres._instance.cursor = connection.cursor()
In a subclass, the `cls' argument would refer to the subclass, while
`Postgres' would still refer to the original class. I have no idea what
this is trying to achieve, and I think it's probably a bug. Maybe
someone else has an idea.
> 3. And is it me or does anyone else think {anydb}.connector’s usage is messy
> and inelegant? Ex:
> print('connecting to PostgreSQL database...')
> connection = Postgres._instance.connection = psycopg2.connect(**db_config)
> cursor = Postgres._instance.cursor = connection.cursor()
> cursor.execute('SELECT VERSION()')
> db_version = cursor.fetchone()
> Why can’t we associate the focus of the connection with the connection
> itself, instead of creating a separate cursor object?
You can have multiple cursors on the same connection.
>
> Also, within the code example, and in Python in general, why does there needs
> to be a __new__ constructor when there’s an __init__ constructor? I’ve read
> about the usage of singletons. It seems you could create singletons within
> __init__ or __new__ . Any enlightenment would be really helpful. I am very
> sleep-deprived, so I’m sorry if this seems like a dumb question.
>
> I have read the official docs. Have also been reading “Python 3: Patterns,
> Recipes, and Idioms”. The first was mildly helpful but I still don’t see the
> benefit of a __new__ constructor.
You can't create a singleton with __init__: by the time __init__ is
called, a new instance has already been created, and you can't switch
the object for a different one. By using __new__, you can bypass the
creation of a new instance.
If what you want is some shared state between all instances, a class
variable or global will do just fine and there's no need for a singleton
instance of anything.
> Why is there dislike for Metaclasses?
They can be confusing.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Single DB connection during class's lifetime. Metaclass, singleton and __new__() examples and references.
On 12/10/2018 01:19, Ryan Johnson wrote:
I am working on using mysql.connector in a class and have found an example of
how to create a single connection that spans the lifetime of all instances of
the class:
https://softwareengineering.stackexchange.com/a/358061/317228
however, I do not understand a few things about the class, including
1. Why it is subclassed as an object: `class Postgres(object):` ? I thought
classes were necessarily objects.
This was sometimes necessary in Python 2.
2. Why is this portion of code directly addressing the class, instead of using
the `cls` reference variable?
connection = Postgres._instance.connection = psycopg2.connect(**db_config)
cursor = Postgres._instance.cursor = connection.cursor()
In a subclass, the `cls' argument would refer to the subclass, while
`Postgres' would still refer to the original class. I have no idea what
this is trying to achieve, and I think it's probably a bug. Maybe
someone else has an idea.
3. And is it me or does anyone else think {anydb}.connector’s usage is messy
and inelegant? Ex:
print('connecting to PostgreSQL database...')
connection = Postgres._instance.connection = psycopg2.connect(**db_config)
cursor = Postgres._instance.cursor = connection.cursor()
cursor.execute('SELECT VERSION()')
db_version = cursor.fetchone()
Why can’t we associate the focus of the connection with the connection
itself, instead of creating a separate cursor object?
You can have multiple cursors on the same connection.
Also, within the code example, and in Python in general, why does there needs
to be a __new__ constructor when there’s an __init__ constructor? I’ve read
about the usage of singletons. It seems you could create singletons within
__init__ or __new__ . Any enlightenment would be really helpful. I am very
sleep-deprived, so I’m sorry if this seems like a dumb question.
I have read the official docs. Have also been reading “Python 3: Patterns,
Recipes, and Idioms”. The first was mildly helpful but I still don’t see the
benefit of a __new__ constructor.
You can't create a singleton with __init__: by the time __init__ is
called, a new instance has already been created, and you can't switch
the object for a different one. By using __new__, you can bypass the
creation of a new instance.
If what you want is some shared state between all instances, a class
variable or global will do just fine and there's no need for a singleton
instance of anything.
Why is there dislike for Metaclasses?
They can be confusing.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Single DB connection during class's lifetime. Metaclass, singleton and __new__() examples and references.
Ryan Johnson writes:
> I am working on using mysql.connector in a class and have found an example of
> how to create a single connection that spans the lifetime of all instances of
> the class:
>
> https://softwareengineering.stackexchange.com/a/358061/317228
>
> however, I do not understand a few things about the class, including
>
> 1. Why it is subclassed as an object: `class Postgres(object):` ? I thought
> classes were necessarily objects.
This obviously is for Python 2.
For historical reasons, Python 2 has two kinds of "class"es,
old style and new style classes. As new style classes were introduced
into Python 2, a need arose to distinguish them from old style classes:
a new style class can be recognized by the fact that is has "object"
in its inheritance graph.
I.e. "class Postgres(object)" indicates that "Postgres" is a new style
class.
Python 3 only supports new style classes. Every class implicitely
inherits "object".
> 2. Why is this portion of code directly addressing the class, instead of
> using the `cls` reference variable?
> connection = Postgres._instance.connection = psycopg2.connect(**db_config)
> cursor = Postgres._instance.cursor = connection.cursor()
There is not real reason to use "Postgres" in this place instead of "cls".
> 3. And is it me or does anyone else think {anydb}.connector’s usage is messy
> and inelegant? Ex:
> print('connecting to PostgreSQL database...')
> connection = Postgres._instance.connection = psycopg2.connect(**db_config)
> cursor = Postgres._instance.cursor = connection.cursor()
> cursor.execute('SELECT VERSION()')
> db_version = cursor.fetchone()
> Why can’t we associate the focus of the connection with the connection
> itself, instead of creating a separate cursor object?
I think this is meant as an example to demonstrate how to
implement singleton database connections. As such, it has
print statements to demonstrates what is going on (i.e.
to convince you that several functions instantiating "postgres" indeed
share a common postgres connection.
In your own productive class, you would likely not have "print" statements.
> Also, within the code example, and in Python in general, why does there needs
> to be a __new__ constructor when there’s an __init__ constructor?
"__new__" essentially allocated the storage for a class instance,
"__init__" initializes this storage. You rarely need "__new__".
Typically, you need it only for classes where the amount
of storage initially allocated for a class instance
depends on the instance's value.
An example is a class inheriting from "tuple": A "tuple" directly
stores references to its components in itself - therefore, the amount
of storage depends on the number of components.
Most classes store the "content" of an instance in an extensible
structure of the instance (the so called "instance dict"). They
can use a standard "__new__" implementation and do not require
there own.
The example you reference could be rewritten to avoid the use of "__new__"
(and integrate its functionality instead into "__init__").
Note that "__new__" is a class method, "__init__" is (by default) an
instance method. This means, the the "cls" in "__new__" corresponds
to "self.__class__" in "__init__".
>I’ve read about the usage of singletons. It seems you could create singletons
>within __init__ or __new__ .
You are right.
--
https://mail.python.org/mailman/listinfo/python-list
