Re: [Freeipa-devel] [PATCH] Allow creation of new connections by unshared instances of backend.Connectible.
Pavel Zuna wrote: Jason Gerard DeRose wrote: On Tue, 2010-01-05 at 14:10 +0100, Pavel Zuna wrote: The backend.Connectible base class was designed, so that only one instance of each subclass is used at a time. Connectible generates a Connection object for each thread and stores it in thread-local storage (context). Subclasses access this object through the Connectible.conn property. This is a good thing, because one instance of the class can be shared by all threads and each thread has its own connection. Unfortunately, this is also a limitation. If a thread needs a second connection (to a different host for example) - it can't do it. Not even by creating a new instance of the Connectible subclass. Ok, let's move from theory to practice: The LDAP backend is currently only used by the Executioner backend, so that plugins can connect to the IPA DS. In the migration plugin, we need a second connection to the DS we're migrating from. The last version had to use low level python-ldap calls to achieve this. In the installer we're still using legacy code from v1. Using ldap2 would be simpler and we could drop ~1000 lines code. (I already started rewriting a few parts to see if it would work.) Proposed solution: Make it possible to create unshared instances of Connectible subclasses. This would be achieved by passing shared_instance=False (couldn't come up with a better name) to the object constructor explicitly. Normally, Connection objects are stored in thread-local storage under the subclass name (e.g. "ldap2"). Unshared instances would store their Connection objects under subclass name + unique instances ID (e.g. "ldap2_218adsfka7"). This is the only solution I could come up with, that doesn't involve breaking a lot of stuff - it just adds a new way of using the code we already have. The attached patches show how it would be done. Pavel I'm fine with this approach as the solution you propose is quite unobtrusive. Is this the final patch then, or will you make further changes or bundle it with another patch? Yeah, this is the final patch. Upcoming patches will depend on it. Pavel pushed to master ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] Allow creation of new connections by unshared instances of backend.Connectible.
Jason Gerard DeRose wrote: On Tue, 2010-01-05 at 14:10 +0100, Pavel Zuna wrote: The backend.Connectible base class was designed, so that only one instance of each subclass is used at a time. Connectible generates a Connection object for each thread and stores it in thread-local storage (context). Subclasses access this object through the Connectible.conn property. This is a good thing, because one instance of the class can be shared by all threads and each thread has its own connection. Unfortunately, this is also a limitation. If a thread needs a second connection (to a different host for example) - it can't do it. Not even by creating a new instance of the Connectible subclass. Ok, let's move from theory to practice: The LDAP backend is currently only used by the Executioner backend, so that plugins can connect to the IPA DS. In the migration plugin, we need a second connection to the DS we're migrating from. The last version had to use low level python-ldap calls to achieve this. In the installer we're still using legacy code from v1. Using ldap2 would be simpler and we could drop ~1000 lines code. (I already started rewriting a few parts to see if it would work.) Proposed solution: Make it possible to create unshared instances of Connectible subclasses. This would be achieved by passing shared_instance=False (couldn't come up with a better name) to the object constructor explicitly. Normally, Connection objects are stored in thread-local storage under the subclass name (e.g. "ldap2"). Unshared instances would store their Connection objects under subclass name + unique instances ID (e.g. "ldap2_218adsfka7"). This is the only solution I could come up with, that doesn't involve breaking a lot of stuff - it just adds a new way of using the code we already have. The attached patches show how it would be done. Pavel I'm fine with this approach as the solution you propose is quite unobtrusive. Is this the final patch then, or will you make further changes or bundle it with another patch? Yeah, this is the final patch. Upcoming patches will depend on it. Pavel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] Allow creation of new connections by unshared instances of backend.Connectible.
On Tue, 2010-01-05 at 14:10 +0100, Pavel Zuna wrote: > The backend.Connectible base class was designed, so that only one instance of > each subclass is used at a time. Connectible generates a Connection object > for > each thread and stores it in thread-local storage (context). Subclasses > access > this object through the Connectible.conn property. > > This is a good thing, because one instance of the class can be shared by all > threads and each thread has its own connection. Unfortunately, this is also a > limitation. If a thread needs a second connection (to a different host for > example) - it can't do it. Not even by creating a new instance of the > Connectible subclass. > > Ok, let's move from theory to practice: > > The LDAP backend is currently only used by the Executioner backend, so that > plugins can connect to the IPA DS. > > In the migration plugin, we need a second connection to the DS we're > migrating > from. The last version had to use low level python-ldap calls to achieve this. > > In the installer we're still using legacy code from v1. Using ldap2 would be > simpler and we could drop ~1000 lines code. (I already started rewriting a > few > parts to see if it would work.) > > Proposed solution: > > Make it possible to create unshared instances of Connectible subclasses. > > This would be achieved by passing shared_instance=False (couldn't come up > with a > better name) to the object constructor explicitly. Normally, Connection > objects > are stored in thread-local storage under the subclass name (e.g. "ldap2"). > Unshared instances would store their Connection objects under subclass name + > unique instances ID (e.g. "ldap2_218adsfka7"). > > This is the only solution I could come up with, that doesn't involve breaking > a > lot of stuff - it just adds a new way of using the code we already have. > > The attached patches show how it would be done. > > Pavel I'm fine with this approach as the solution you propose is quite unobtrusive. Is this the final patch then, or will you make further changes or bundle it with another patch? ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] Allow creation of new connections by unshared instances of backend.Connectible.
The backend.Connectible base class was designed, so that only one instance of each subclass is used at a time. Connectible generates a Connection object for each thread and stores it in thread-local storage (context). Subclasses access this object through the Connectible.conn property. This is a good thing, because one instance of the class can be shared by all threads and each thread has its own connection. Unfortunately, this is also a limitation. If a thread needs a second connection (to a different host for example) - it can't do it. Not even by creating a new instance of the Connectible subclass. Ok, let's move from theory to practice: The LDAP backend is currently only used by the Executioner backend, so that plugins can connect to the IPA DS. In the migration plugin, we need a second connection to the DS we're migrating from. The last version had to use low level python-ldap calls to achieve this. In the installer we're still using legacy code from v1. Using ldap2 would be simpler and we could drop ~1000 lines code. (I already started rewriting a few parts to see if it would work.) Proposed solution: Make it possible to create unshared instances of Connectible subclasses. This would be achieved by passing shared_instance=False (couldn't come up with a better name) to the object constructor explicitly. Normally, Connection objects are stored in thread-local storage under the subclass name (e.g. "ldap2"). Unshared instances would store their Connection objects under subclass name + unique instances ID (e.g. "ldap2_218adsfka7"). This is the only solution I could come up with, that doesn't involve breaking a lot of stuff - it just adds a new way of using the code we already have. The attached patches show how it would be done. Pavel 0001-Allow-creation-of-new-connections-by-unshared-instan.patch Description: application/mbox ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
