Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

Yes, it works.
I'm grateful for your help :)

ср, 25 окт. 2017 г. в 16:56, Mike Bayer :

> On Wed, Oct 25, 2017 at 9:25 AM, Антонио Антуан 
> wrote:
> > As I said befire, sqlalchemy version: 1.0.19 and code is here:
> > https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>
> I apologize, did not see that link.
>
> Use the execute_and_instances() implemenation from
> lib/sqlalchemy/orm/query.py, which means here use _bind_mapper():
>
> def _execute_and_instances(self, querycontext):
> self._check_bound_to_shard()
> querycontext.attributes['shard_id'] = self._shard_id
> result = self._connection_from_session(
> mapper=self._bind_mapper(),
> shard_id=self._shard_id).execute(
> querycontext.statement,
> self._params
> )
> return self.instances(result, querycontext)
>
>
>
>
> >
> > Currently I can't use another version. In that case, I think that here is
> > the right decision:
> > "Looks like I have to check if "_mapper_zero()" returns real mapper.
> > Otherwise I should pass "None" to "_connection_from_session()" as value
> of
> > "mapper" argument"
> >
> > Am I right?
> >
> >
> > ср, 25 окт. 2017 г. в 16:18, Mike Bayer :
> >>
> >> On Wed, Oct 25, 2017 at 6:52 AM, Антонио Антуан 
> >> wrote:
> >> > Any news here?
> >>
> >> We would require a self-contained demonstration case that illustrates
> >> your error as well as complete information on what specific SQLAlchemy
> >> version you are targeting.  As mentioned in a different reply,
> >> _mapper_zero has been fixed in 1.1 to always return a mapper.  I
> >> would strongly advise targeting the 1.1 series for any new development
> >> as that's where issues can still be fixed.
> >>
> >>
> >>
> >>
> >>
> >> >
> >> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио
> Антуан
> >> > написал:
> >> >>
> >> >> I see that it is not happened when "bind" passed directly to
> >> >> "sessionmaker"
> >> >>
> >> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
> >> >>>
> >> >>>
> >> >>>
> >> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
> >> >>> написал:
> >> 
> >>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King <
> si...@simonking.org.uk>
> >>  wrote:
> >>  > The "is not None" is important when checking a variable that may
> >>  > contain a ClauseElement, precisely because ClauseElement defines
> >>  > that
> >>  > __bool__ method.
> >>  >
> >>  > However, in Session.get_bind(), "mapper" is not supposed to
> contain
> >>  > a
> >>  > ClauseElement. It should either be an instance of
> >>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
> >>  > is a
> >>  > valid and concise way to write it. Comparing to None might be
> >>  > strictly
> >>  > more accurate, but it shouldn't be necessary.
> >> 
> >>  agree, "mapper" means "mapper" and it is not supposed to be a
> >>  ClauseElement.   This sounds like arguments are not being passed
> >>  correctly somewhere.   Would need a demonstration script if
> something
> >>  in SQLAlchemy itself is claimed to be making the mistake.
> >> >>>
> >> >>>
> >> >>> Ok, that code produces mentioned error:
> >> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
> >> >>> Traceback (most recent call last):
> >> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
> >> >>> 
> >> >>> Session.query(with_recursive).set_shard('default').all()
> >> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> >> >>> 2654,
> >> >>> in all
> >> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> >> >>> 2802,
> >> >>> in __iter__
> >> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
> >> >>> _execute_and_instances
> >> >>> shard_id=self._shard_id).execute(
> >> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> >> >>> 2806,
> >> >>> in _connection_from_session
> >> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py",
> line
> >> >>> 984, in connection
> >> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
> >> >>> get_bind
> >> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
> >> >>> clause)
> >> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py",
> line
> >> >>> 1336, in get_bind
> >> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py",
> line
> >> >>> 539, in __bool__
> >> >>> TypeError: Boolean value of this clause is not defined
> >> >>>
> >> 
> >>  >
> >>  > Simon
> >> 
> >>  >
> >>  >
> >>  >
> >>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан <
> a.ch...@gmail.com>
> >>  > wrote:
> >>  >> Really, `mapper` contains this:  at
> >>  >> 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Mike Bayer]

On Wed, Oct 25, 2017 at 9:25 AM, Антонио Антуан  wrote:
> As I said befire, sqlalchemy version: 1.0.19 and code is here:
> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f

I apologize, did not see that link.

Use the execute_and_instances() implemenation from
lib/sqlalchemy/orm/query.py, which means here use _bind_mapper():

def _execute_and_instances(self, querycontext):
self._check_bound_to_shard()
querycontext.attributes['shard_id'] = self._shard_id
result = self._connection_from_session(
mapper=self._bind_mapper(),
shard_id=self._shard_id).execute(
querycontext.statement,
self._params
)
return self.instances(result, querycontext)




>
> Currently I can't use another version. In that case, I think that here is
> the right decision:
> "Looks like I have to check if "_mapper_zero()" returns real mapper.
> Otherwise I should pass "None" to "_connection_from_session()" as value of
> "mapper" argument"
>
> Am I right?
>
>
> ср, 25 окт. 2017 г. в 16:18, Mike Bayer :
>>
>> On Wed, Oct 25, 2017 at 6:52 AM, Антонио Антуан 
>> wrote:
>> > Any news here?
>>
>> We would require a self-contained demonstration case that illustrates
>> your error as well as complete information on what specific SQLAlchemy
>> version you are targeting.  As mentioned in a different reply,
>> _mapper_zero has been fixed in 1.1 to always return a mapper.  I
>> would strongly advise targeting the 1.1 series for any new development
>> as that's where issues can still be fixed.
>>
>>
>>
>>
>>
>> >
>> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
>> > написал:
>> >>
>> >> I see that it is not happened when "bind" passed directly to
>> >> "sessionmaker"
>> >>
>> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>> >>>
>> >>>
>> >>>
>> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
>> >>> написал:
>> 
>>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King 
>>  wrote:
>>  > The "is not None" is important when checking a variable that may
>>  > contain a ClauseElement, precisely because ClauseElement defines
>>  > that
>>  > __bool__ method.
>>  >
>>  > However, in Session.get_bind(), "mapper" is not supposed to contain
>>  > a
>>  > ClauseElement. It should either be an instance of
>>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
>>  > is a
>>  > valid and concise way to write it. Comparing to None might be
>>  > strictly
>>  > more accurate, but it shouldn't be necessary.
>> 
>>  agree, "mapper" means "mapper" and it is not supposed to be a
>>  ClauseElement.   This sounds like arguments are not being passed
>>  correctly somewhere.   Would need a demonstration script if something
>>  in SQLAlchemy itself is claimed to be making the mistake.
>> >>>
>> >>>
>> >>> Ok, that code produces mentioned error:
>> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>> >>> Traceback (most recent call last):
>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
>> >>> 
>> >>> Session.query(with_recursive).set_shard('default').all()
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>> >>> 2654,
>> >>> in all
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>> >>> 2802,
>> >>> in __iter__
>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
>> >>> _execute_and_instances
>> >>> shard_id=self._shard_id).execute(
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>> >>> 2806,
>> >>> in _connection_from_session
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>> >>> 984, in connection
>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
>> >>> get_bind
>> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
>> >>> clause)
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>> >>> 1336, in get_bind
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
>> >>> 539, in __bool__
>> >>> TypeError: Boolean value of this clause is not defined
>> >>>
>> 
>>  >
>>  > Simon
>> 
>>  >
>>  >
>>  >
>>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан 
>>  > wrote:
>>  >> Really, `mapper` contains this: >  >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>>  >> already
>>  >> encountered this problem in my code, when I checked sqla-objects
>>  >> existance
>>  >> without "is not None", project was broken. Is it normal to omit
>>  >> that
>>  >> condition in sqlalchemy code?
>>  >>
>>  >> I use:
>>  > sqlalchemy.__version__
>>  >> '1.0.19'

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

As I said befire, sqlalchemy version: 1.0.19 and code is here:
https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f

Currently I can't use another version. In that case, I think that here is
the right decision:
"Looks like I have to check if "_mapper_zero()" returns real mapper.
Otherwise I should pass "None" to "_connection_from_session()" as value of
"mapper" argument"

Am I right?


ср, 25 окт. 2017 г. в 16:18, Mike Bayer :

> On Wed, Oct 25, 2017 at 6:52 AM, Антонио Антуан 
> wrote:
> > Any news here?
>
> We would require a self-contained demonstration case that illustrates
> your error as well as complete information on what specific SQLAlchemy
> version you are targeting.  As mentioned in a different reply,
> _mapper_zero has been fixed in 1.1 to always return a mapper.  I
> would strongly advise targeting the 1.1 series for any new development
> as that's where issues can still be fixed.
>
>
>
>
>
> >
> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
> > написал:
> >>
> >> I see that it is not happened when "bind" passed directly to
> >> "sessionmaker"
> >>
> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
> >>>
> >>>
> >>>
> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
> >>> написал:
> 
>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King 
>  wrote:
>  > The "is not None" is important when checking a variable that may
>  > contain a ClauseElement, precisely because ClauseElement defines
> that
>  > __bool__ method.
>  >
>  > However, in Session.get_bind(), "mapper" is not supposed to contain
> a
>  > ClauseElement. It should either be an instance of
>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
> is a
>  > valid and concise way to write it. Comparing to None might be
> strictly
>  > more accurate, but it shouldn't be necessary.
> 
>  agree, "mapper" means "mapper" and it is not supposed to be a
>  ClauseElement.   This sounds like arguments are not being passed
>  correctly somewhere.   Would need a demonstration script if something
>  in SQLAlchemy itself is claimed to be making the mistake.
> >>>
> >>>
> >>> Ok, that code produces mentioned error:
> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
> >>> Traceback (most recent call last):
> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
> >>> 
> >>> Session.query(with_recursive).set_shard('default').all()
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> 2654,
> >>> in all
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> 2802,
> >>> in __iter__
> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
> >>> _execute_and_instances
> >>> shard_id=self._shard_id).execute(
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> 2806,
> >>> in _connection_from_session
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
> >>> 984, in connection
> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
> >>> get_bind
> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
> clause)
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
> >>> 1336, in get_bind
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
> >>> 539, in __bool__
> >>> TypeError: Boolean value of this clause is not defined
> >>>
> 
>  >
>  > Simon
> 
>  >
>  >
>  >
>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан 
>  > wrote:
>  >> Really, `mapper` contains this:   >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>  >> already
>  >> encountered this problem in my code, when I checked sqla-objects
>  >> existance
>  >> without "is not None", project was broken. Is it normal to omit
> that
>  >> condition in sqlalchemy code?
>  >>
>  >> I use:
>  > sqlalchemy.__version__
>  >> '1.0.19'
>  >>
>  >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King
>  >> написал:
>  >>>
>  >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан <
> a.ch...@gmail.com>
>  >>> wrote:
>  >>> > Hi.
>  >>> > I use my own `RoutingSession` and `RoutingQuery` implementation,
>  >>> > most of
>  >>> > it
>  >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>  >>> >
>  >>> > class RoutingSession(Session):
>  >>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
>  >>> > **kwargs):
>  >>> > original_bind = None
>  >>> > try:
>  >>> > original_bind = super(RoutingSession,
>  >>> > self).get_bind(mapper,
>  >>> > clause)
>  >>> > except UnboundExecutionError:
>  >>> >

Re: [sqlalchemy] mapper existance checks possibly wrong

[Mike Bayer]

On Wed, Oct 25, 2017 at 6:52 AM, Антонио Антуан  wrote:
> Any news here?

We would require a self-contained demonstration case that illustrates
your error as well as complete information on what specific SQLAlchemy
version you are targeting.  As mentioned in a different reply,
_mapper_zero has been fixed in 1.1 to always return a mapper.  I
would strongly advise targeting the 1.1 series for any new development
as that's where issues can still be fixed.





>
> суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
> написал:
>>
>> I see that it is not happened when "bind" passed directly to
>> "sessionmaker"
>>
>> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>>>
>>>
>>>
>>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
>>> написал:

 On Fri, Oct 20, 2017 at 11:05 AM, Simon King 
 wrote:
 > The "is not None" is important when checking a variable that may
 > contain a ClauseElement, precisely because ClauseElement defines that
 > __bool__ method.
 >
 > However, in Session.get_bind(), "mapper" is not supposed to contain a
 > ClauseElement. It should either be an instance of
 > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a
 > valid and concise way to write it. Comparing to None might be strictly
 > more accurate, but it shouldn't be necessary.

 agree, "mapper" means "mapper" and it is not supposed to be a
 ClauseElement.   This sounds like arguments are not being passed
 correctly somewhere.   Would need a demonstration script if something
 in SQLAlchemy itself is claimed to be making the mistake.
>>>
>>>
>>> Ok, that code produces mentioned error:
>>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>>> Traceback (most recent call last):
>>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
>>> 
>>> Session.query(with_recursive).set_shard('default').all()
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2654,
>>> in all
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2802,
>>> in __iter__
>>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
>>> _execute_and_instances
>>> shard_id=self._shard_id).execute(
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2806,
>>> in _connection_from_session
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> 984, in connection
>>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
>>> get_bind
>>> original_bind = super(RoutingSession, self).get_bind(mapper, clause)
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> 1336, in get_bind
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
>>> 539, in __bool__
>>> TypeError: Boolean value of this clause is not defined
>>>

 >
 > Simon

 >
 >
 >
 > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан 
 > wrote:
 >> Really, `mapper` contains this: >>> >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
 >> already
 >> encountered this problem in my code, when I checked sqla-objects
 >> existance
 >> without "is not None", project was broken. Is it normal to omit that
 >> condition in sqlalchemy code?
 >>
 >> I use:
 > sqlalchemy.__version__
 >> '1.0.19'
 >>
 >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King
 >> написал:
 >>>
 >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан 
 >>> wrote:
 >>> > Hi.
 >>> > I use my own `RoutingSession` and `RoutingQuery` implementation,
 >>> > most of
 >>> > it
 >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
 >>> >
 >>> > class RoutingSession(Session):
 >>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
 >>> > **kwargs):
 >>> > original_bind = None
 >>> > try:
 >>> > original_bind = super(RoutingSession,
 >>> > self).get_bind(mapper,
 >>> > clause)
 >>> > except UnboundExecutionError:
 >>> > # may not be bound
 >>> > pass
 >>> > if shard_id is None:
 >>> > shard_id = TenantIDStorage.get_shard_id()  # just
 >>> > global
 >>> > storage
 >>> > bind_for_shard = self.__binds[shard_id]
 >>> > if original_bind is not None and original_bind.url ==
 >>> > bind_for_shard.url:
 >>> > return original_bind
 >>> > else:
 >>> > return bind_for_shard
 >>> >
 >>> > def __init__(self, shards=None, query_cls=CachingQuery,
 >>> > engines_factory=None, **kwargs):
 >>> > super(RoutingSession, self).__init__(query_cls=query_cls,
 >>> > **kwargs)
 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Mike Bayer]

On Wed, Oct 25, 2017 at 8:02 AM, Антонио Антуан  wrote:
> Looks like I have to check if _mapper_zero() returns real mapper.

 _mapper_zero() in the 1.1 and 1.2 series will never return a
non-mapper (see https://bitbucket.org/zzzeek/sqlalchemy/issues/3608).
 I would strongly advise working with the 1.1. series at least as
1.0.x. is security patches only.



Otherwise
> I should pass None to "_connection_from_session()" as value of "mapper"
> argument. Right?
>
> ср, 25 окт. 2017 г. в 15:00, Антонио Антуан :
>>
>> As I mentioned before, "> group_getter>".
>>
>> ср, 25 окт. 2017 г. в 14:19, Simon King :
>>>
>>> What does self._mapper_zero() return in your
>>> RoutingQuery._execute_and_instances method?
>>>
>>> Simon
>>>
>>> On Wed, Oct 25, 2017 at 11:52 AM, Антонио Антуан 
>>> wrote:
>>> > Any news here?
>>> >
>>> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
>>> > написал:
>>> >>
>>> >> I see that it is not happened when "bind" passed directly to
>>> >> "sessionmaker"
>>> >>
>>> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>>> >>>
>>> >>>
>>> >>>
>>> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
>>> >>> написал:
>>> 
>>>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King
>>>  
>>>  wrote:
>>>  > The "is not None" is important when checking a variable that may
>>>  > contain a ClauseElement, precisely because ClauseElement defines
>>>  > that
>>>  > __bool__ method.
>>>  >
>>>  > However, in Session.get_bind(), "mapper" is not supposed to
>>>  > contain a
>>>  > ClauseElement. It should either be an instance of
>>>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
>>>  > is a
>>>  > valid and concise way to write it. Comparing to None might be
>>>  > strictly
>>>  > more accurate, but it shouldn't be necessary.
>>> 
>>>  agree, "mapper" means "mapper" and it is not supposed to be a
>>>  ClauseElement.   This sounds like arguments are not being passed
>>>  correctly somewhere.   Would need a demonstration script if
>>>  something
>>>  in SQLAlchemy itself is claimed to be making the mistake.
>>> >>>
>>> >>>
>>> >>> Ok, that code produces mentioned error:
>>> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>>> >>> Traceback (most recent call last):
>>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
>>> >>> 
>>> >>> Session.query(with_recursive).set_shard('default').all()
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>>> >>> 2654,
>>> >>> in all
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>>> >>> 2802,
>>> >>> in __iter__
>>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
>>> >>> _execute_and_instances
>>> >>> shard_id=self._shard_id).execute(
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>>> >>> 2806,
>>> >>> in _connection_from_session
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> >>> 984, in connection
>>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
>>> >>> get_bind
>>> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
>>> >>> clause)
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> >>> 1336, in get_bind
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py",
>>> >>> line
>>> >>> 539, in __bool__
>>> >>> TypeError: Boolean value of this clause is not defined
>>> >>>
>>> 
>>>  >
>>>  > Simon
>>> 
>>>  >
>>>  >
>>>  >
>>>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан
>>>  > 
>>>  > wrote:
>>>  >> Really, `mapper` contains this: >>  >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>>>  >> already
>>>  >> encountered this problem in my code, when I checked sqla-objects
>>>  >> existance
>>>  >> without "is not None", project was broken. Is it normal to omit
>>>  >> that
>>>  >> condition in sqlalchemy code?
>>>  >>
>>>  >> I use:
>>>  > sqlalchemy.__version__
>>>  >> '1.0.19'
>>>  >>
>>>  >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon
>>>  >> King
>>>  >> написал:
>>>  >>>
>>>  >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан
>>>  >>> 
>>>  >>> wrote:
>>>  >>> > Hi.
>>>  >>> > I use my own `RoutingSession` and `RoutingQuery`
>>>  >>> > implementation,
>>>  >>> > most of
>>>  >>> > it
>>>  >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>>>  >>> >
>>>  >>> > class RoutingSession(Session):
>>>  >>> > def get_bind(self, mapper=None, clause=None,
>>>  >>> > shard_id=None,
>>>  >>> > 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Simon King]

This is complete guesswork - I haven't examined this code in detail so
I could be wrong. Also, I don't know what version of SQLAlchemy you
are using, so this may not apply.

Session.connection takes separate "mapper" and "clause" parameters:

https://bitbucket.org/zzzeek/sqlalchemy/src/f34b180ca9059a74c3bf1db1b79e187c3f4b81c9/lib/sqlalchemy/orm/session.py#session.py-958

Query._execute_and_instances passes both of those parameters:

https://bitbucket.org/zzzeek/sqlalchemy/src/f34b180ca9059a74c3bf1db1b79e187c3f4b81c9/lib/sqlalchemy/orm/query.py#query.py-2893

(via the _get_bind_args helper function)

Your overridden version of _execute_and_instances is passing a clause
in the "mapper" parameter, which seems wrong. Somehow you need to fix
that, perhaps by using the _get_bind_args helper.

Simon



On Wed, Oct 25, 2017 at 1:02 PM, Антонио Антуан  wrote:
> Looks like I have to check if _mapper_zero() returns real mapper. Otherwise
> I should pass None to "_connection_from_session()" as value of "mapper"
> argument. Right?
>
> ср, 25 окт. 2017 г. в 15:00, Антонио Антуан :
>>
>> As I mentioned before, "> group_getter>".
>>
>> ср, 25 окт. 2017 г. в 14:19, Simon King :
>>>
>>> What does self._mapper_zero() return in your
>>> RoutingQuery._execute_and_instances method?
>>>
>>> Simon
>>>
>>> On Wed, Oct 25, 2017 at 11:52 AM, Антонио Антуан 
>>> wrote:
>>> > Any news here?
>>> >
>>> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
>>> > написал:
>>> >>
>>> >> I see that it is not happened when "bind" passed directly to
>>> >> "sessionmaker"
>>> >>
>>> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>>> >>>
>>> >>>
>>> >>>
>>> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
>>> >>> написал:
>>> 
>>>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King
>>>  
>>>  wrote:
>>>  > The "is not None" is important when checking a variable that may
>>>  > contain a ClauseElement, precisely because ClauseElement defines
>>>  > that
>>>  > __bool__ method.
>>>  >
>>>  > However, in Session.get_bind(), "mapper" is not supposed to
>>>  > contain a
>>>  > ClauseElement. It should either be an instance of
>>>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
>>>  > is a
>>>  > valid and concise way to write it. Comparing to None might be
>>>  > strictly
>>>  > more accurate, but it shouldn't be necessary.
>>> 
>>>  agree, "mapper" means "mapper" and it is not supposed to be a
>>>  ClauseElement.   This sounds like arguments are not being passed
>>>  correctly somewhere.   Would need a demonstration script if
>>>  something
>>>  in SQLAlchemy itself is claimed to be making the mistake.
>>> >>>
>>> >>>
>>> >>> Ok, that code produces mentioned error:
>>> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>>> >>> Traceback (most recent call last):
>>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
>>> >>> 
>>> >>> Session.query(with_recursive).set_shard('default').all()
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>>> >>> 2654,
>>> >>> in all
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>>> >>> 2802,
>>> >>> in __iter__
>>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
>>> >>> _execute_and_instances
>>> >>> shard_id=self._shard_id).execute(
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>>> >>> 2806,
>>> >>> in _connection_from_session
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> >>> 984, in connection
>>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
>>> >>> get_bind
>>> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
>>> >>> clause)
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> >>> 1336, in get_bind
>>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py",
>>> >>> line
>>> >>> 539, in __bool__
>>> >>> TypeError: Boolean value of this clause is not defined
>>> >>>
>>> 
>>>  >
>>>  > Simon
>>> 
>>>  >
>>>  >
>>>  >
>>>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан
>>>  > 
>>>  > wrote:
>>>  >> Really, `mapper` contains this: >>  >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>>>  >> already
>>>  >> encountered this problem in my code, when I checked sqla-objects
>>>  >> existance
>>>  >> without "is not None", project was broken. Is it normal to omit
>>>  >> that
>>>  >> condition in sqlalchemy code?
>>>  >>
>>>  >> I use:
>>>  > sqlalchemy.__version__
>>>  >> '1.0.19'
>>>  >>
>>>  >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

Looks like I have to check if _mapper_zero() returns real mapper. Otherwise
I should pass None to "_connection_from_session()" as value of "mapper"
argument. Right?

ср, 25 окт. 2017 г. в 15:00, Антонио Антуан :

> As I mentioned before, " group_getter>".
>
> ср, 25 окт. 2017 г. в 14:19, Simon King :
>
>> What does self._mapper_zero() return in your
>> RoutingQuery._execute_and_instances method?
>>
>> Simon
>>
>> On Wed, Oct 25, 2017 at 11:52 AM, Антонио Антуан 
>> wrote:
>> > Any news here?
>> >
>> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
>> > написал:
>> >>
>> >> I see that it is not happened when "bind" passed directly to
>> >> "sessionmaker"
>> >>
>> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>> >>>
>> >>>
>> >>>
>> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
>> >>> написал:
>> 
>>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King > >
>>  wrote:
>>  > The "is not None" is important when checking a variable that may
>>  > contain a ClauseElement, precisely because ClauseElement defines
>> that
>>  > __bool__ method.
>>  >
>>  > However, in Session.get_bind(), "mapper" is not supposed to
>> contain a
>>  > ClauseElement. It should either be an instance of
>>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
>> is a
>>  > valid and concise way to write it. Comparing to None might be
>> strictly
>>  > more accurate, but it shouldn't be necessary.
>> 
>>  agree, "mapper" means "mapper" and it is not supposed to be a
>>  ClauseElement.   This sounds like arguments are not being passed
>>  correctly somewhere.   Would need a demonstration script if something
>>  in SQLAlchemy itself is claimed to be making the mistake.
>> >>>
>> >>>
>> >>> Ok, that code produces mentioned error:
>> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>> >>> Traceback (most recent call last):
>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
>> >>> 
>> >>> Session.query(with_recursive).set_shard('default').all()
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>> 2654,
>> >>> in all
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>> 2802,
>> >>> in __iter__
>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
>> >>> _execute_and_instances
>> >>> shard_id=self._shard_id).execute(
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
>> 2806,
>> >>> in _connection_from_session
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>> >>> 984, in connection
>> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
>> >>> get_bind
>> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
>> clause)
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>> >>> 1336, in get_bind
>> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
>> >>> 539, in __bool__
>> >>> TypeError: Boolean value of this clause is not defined
>> >>>
>> 
>>  >
>>  > Simon
>> 
>>  >
>>  >
>>  >
>>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан > >
>>  > wrote:
>>  >> Really, `mapper` contains this: >  >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>>  >> already
>>  >> encountered this problem in my code, when I checked sqla-objects
>>  >> existance
>>  >> without "is not None", project was broken. Is it normal to omit
>> that
>>  >> condition in sqlalchemy code?
>>  >>
>>  >> I use:
>>  > sqlalchemy.__version__
>>  >> '1.0.19'
>>  >>
>>  >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon
>> King
>>  >> написал:
>>  >>>
>>  >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан <
>> a.ch...@gmail.com>
>>  >>> wrote:
>>  >>> > Hi.
>>  >>> > I use my own `RoutingSession` and `RoutingQuery`
>> implementation,
>>  >>> > most of
>>  >>> > it
>>  >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>>  >>> >
>>  >>> > class RoutingSession(Session):
>>  >>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
>>  >>> > **kwargs):
>>  >>> > original_bind = None
>>  >>> > try:
>>  >>> > original_bind = super(RoutingSession,
>>  >>> > self).get_bind(mapper,
>>  >>> > clause)
>>  >>> > except UnboundExecutionError:
>>  >>> > # may not be bound
>>  >>> > pass
>>  >>> > if shard_id is None:
>>  >>> > shard_id = TenantIDStorage.get_shard_id()  # just
>>  >>> > global
>>  >>> > storage
>>  >>> > bind_for_shard = self.__binds[shard_id]
>>  

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

As I mentioned before, "".

ср, 25 окт. 2017 г. в 14:19, Simon King :

> What does self._mapper_zero() return in your
> RoutingQuery._execute_and_instances method?
>
> Simon
>
> On Wed, Oct 25, 2017 at 11:52 AM, Антонио Антуан 
> wrote:
> > Any news here?
> >
> > суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
> > написал:
> >>
> >> I see that it is not happened when "bind" passed directly to
> >> "sessionmaker"
> >>
> >> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
> >>>
> >>>
> >>>
> >>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
> >>> написал:
> 
>  On Fri, Oct 20, 2017 at 11:05 AM, Simon King 
>  wrote:
>  > The "is not None" is important when checking a variable that may
>  > contain a ClauseElement, precisely because ClauseElement defines
> that
>  > __bool__ method.
>  >
>  > However, in Session.get_bind(), "mapper" is not supposed to contain
> a
>  > ClauseElement. It should either be an instance of
>  > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:"
> is a
>  > valid and concise way to write it. Comparing to None might be
> strictly
>  > more accurate, but it shouldn't be necessary.
> 
>  agree, "mapper" means "mapper" and it is not supposed to be a
>  ClauseElement.   This sounds like arguments are not being passed
>  correctly somewhere.   Would need a demonstration script if something
>  in SQLAlchemy itself is claimed to be making the mistake.
> >>>
> >>>
> >>> Ok, that code produces mentioned error:
> >>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
> >>> Traceback (most recent call last):
> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
> >>> 
> >>> Session.query(with_recursive).set_shard('default').all()
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> 2654,
> >>> in all
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> 2802,
> >>> in __iter__
> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
> >>> _execute_and_instances
> >>> shard_id=self._shard_id).execute(
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line
> 2806,
> >>> in _connection_from_session
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
> >>> 984, in connection
> >>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
> >>> get_bind
> >>> original_bind = super(RoutingSession, self).get_bind(mapper,
> clause)
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
> >>> 1336, in get_bind
> >>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
> >>> 539, in __bool__
> >>> TypeError: Boolean value of this clause is not defined
> >>>
> 
>  >
>  > Simon
> 
>  >
>  >
>  >
>  > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан 
>  > wrote:
>  >> Really, `mapper` contains this:   >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>  >> already
>  >> encountered this problem in my code, when I checked sqla-objects
>  >> existance
>  >> without "is not None", project was broken. Is it normal to omit
> that
>  >> condition in sqlalchemy code?
>  >>
>  >> I use:
>  > sqlalchemy.__version__
>  >> '1.0.19'
>  >>
>  >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King
>  >> написал:
>  >>>
>  >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан <
> a.ch...@gmail.com>
>  >>> wrote:
>  >>> > Hi.
>  >>> > I use my own `RoutingSession` and `RoutingQuery` implementation,
>  >>> > most of
>  >>> > it
>  >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>  >>> >
>  >>> > class RoutingSession(Session):
>  >>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
>  >>> > **kwargs):
>  >>> > original_bind = None
>  >>> > try:
>  >>> > original_bind = super(RoutingSession,
>  >>> > self).get_bind(mapper,
>  >>> > clause)
>  >>> > except UnboundExecutionError:
>  >>> > # may not be bound
>  >>> > pass
>  >>> > if shard_id is None:
>  >>> > shard_id = TenantIDStorage.get_shard_id()  # just
>  >>> > global
>  >>> > storage
>  >>> > bind_for_shard = self.__binds[shard_id]
>  >>> > if original_bind is not None and original_bind.url ==
>  >>> > bind_for_shard.url:
>  >>> > return original_bind
>  >>> > else:
>  >>> > return bind_for_shard
>  >>> >
>  >>> > def __init__(self, shards=None, query_cls=CachingQuery,
>  >>> > engines_factory=None, **kwargs):
>  >>> > 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Simon King]

What does self._mapper_zero() return in your
RoutingQuery._execute_and_instances method?

Simon

On Wed, Oct 25, 2017 at 11:52 AM, Антонио Антуан  wrote:
> Any news here?
>
> суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан
> написал:
>>
>> I see that it is not happened when "bind" passed directly to
>> "sessionmaker"
>>
>> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>>>
>>>
>>>
>>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
>>> написал:

 On Fri, Oct 20, 2017 at 11:05 AM, Simon King 
 wrote:
 > The "is not None" is important when checking a variable that may
 > contain a ClauseElement, precisely because ClauseElement defines that
 > __bool__ method.
 >
 > However, in Session.get_bind(), "mapper" is not supposed to contain a
 > ClauseElement. It should either be an instance of
 > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a
 > valid and concise way to write it. Comparing to None might be strictly
 > more accurate, but it shouldn't be necessary.

 agree, "mapper" means "mapper" and it is not supposed to be a
 ClauseElement.   This sounds like arguments are not being passed
 correctly somewhere.   Would need a demonstration script if something
 in SQLAlchemy itself is claimed to be making the mistake.
>>>
>>>
>>> Ok, that code produces mentioned error:
>>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>>> Traceback (most recent call last):
>>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in
>>> 
>>> Session.query(with_recursive).set_shard('default').all()
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2654,
>>> in all
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2802,
>>> in __iter__
>>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
>>> _execute_and_instances
>>> shard_id=self._shard_id).execute(
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2806,
>>> in _connection_from_session
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> 984, in connection
>>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in
>>> get_bind
>>> original_bind = super(RoutingSession, self).get_bind(mapper, clause)
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
>>> 1336, in get_bind
>>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
>>> 539, in __bool__
>>> TypeError: Boolean value of this clause is not defined
>>>

 >
 > Simon

 >
 >
 >
 > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан 
 > wrote:
 >> Really, `mapper` contains this: >>> >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
 >> already
 >> encountered this problem in my code, when I checked sqla-objects
 >> existance
 >> without "is not None", project was broken. Is it normal to omit that
 >> condition in sqlalchemy code?
 >>
 >> I use:
 > sqlalchemy.__version__
 >> '1.0.19'
 >>
 >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King
 >> написал:
 >>>
 >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан 
 >>> wrote:
 >>> > Hi.
 >>> > I use my own `RoutingSession` and `RoutingQuery` implementation,
 >>> > most of
 >>> > it
 >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
 >>> >
 >>> > class RoutingSession(Session):
 >>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
 >>> > **kwargs):
 >>> > original_bind = None
 >>> > try:
 >>> > original_bind = super(RoutingSession,
 >>> > self).get_bind(mapper,
 >>> > clause)
 >>> > except UnboundExecutionError:
 >>> > # may not be bound
 >>> > pass
 >>> > if shard_id is None:
 >>> > shard_id = TenantIDStorage.get_shard_id()  # just
 >>> > global
 >>> > storage
 >>> > bind_for_shard = self.__binds[shard_id]
 >>> > if original_bind is not None and original_bind.url ==
 >>> > bind_for_shard.url:
 >>> > return original_bind
 >>> > else:
 >>> > return bind_for_shard
 >>> >
 >>> > def __init__(self, shards=None, query_cls=CachingQuery,
 >>> > engines_factory=None, **kwargs):
 >>> > super(RoutingSession, self).__init__(query_cls=query_cls,
 >>> > **kwargs)
 >>> > self.__binds = {}
 >>> > self.engines_factory = engines_factory
 >>> > if shards is not None:
 >>> > self.update_shards(**shards)
 >>> >
 >>> > def _add_bind(self, key, bind):
 >>> > self.__binds[key] = bind
 >>> 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

Any news here?

суббота, 21 октября 2017 г., 18:42:47 UTC+3 пользователь Антонио Антуан 
написал:
>
> I see that it is not happened when "bind" passed directly to 
> "sessionmaker" 
>
> сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :
>
>>
>>
>> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer 
>> написал:
>>
>>> On Fri, Oct 20, 2017 at 11:05 AM, Simon King  
>>> wrote: 
>>> > The "is not None" is important when checking a variable that may 
>>> > contain a ClauseElement, precisely because ClauseElement defines that 
>>> > __bool__ method. 
>>> > 
>>> > However, in Session.get_bind(), "mapper" is not supposed to contain a 
>>> > ClauseElement. It should either be an instance of 
>>> > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a 
>>> > valid and concise way to write it. Comparing to None might be strictly 
>>> > more accurate, but it shouldn't be necessary. 
>>>
>>> agree, "mapper" means "mapper" and it is not supposed to be a 
>>> ClauseElement.   This sounds like arguments are not being passed 
>>> correctly somewhere.   Would need a demonstration script if something 
>>> in SQLAlchemy itself is claimed to be making the mistake. 
>>>
>>
>> Ok, that code produces mentioned error: 
>> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
>> Traceback (most recent call last):
>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in 
>> 
>> Session.query(with_recursive).set_shard('default').all()
>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2654, 
>> in all
>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2802, 
>> in __iter__
>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in 
>> _execute_and_instances
>> shard_id=self._shard_id).execute(
>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2806, 
>> in _connection_from_session
>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 
>> 984, in connection
>>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in get_bind
>> original_bind = super(RoutingSession, self).get_bind(mapper, clause)
>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 
>> 1336, in get_bind
>>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line 
>> 539, in __bool__
>> TypeError: Boolean value of this clause is not defined
>>  
>>
>>> > 
>>> > Simon 
>>>
>> > 
>>> > 
>>> > 
>>> > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан  
>>> wrote: 
>>> >> Really, `mapper` contains this: >> >> 0x7fe673fb0a50; group_getter>. But does it changes something? I 
>>> already 
>>> >> encountered this problem in my code, when I checked sqla-objects 
>>> existance 
>>> >> without "is not None", project was broken. Is it normal to omit that 
>>> >> condition in sqlalchemy code? 
>>> >> 
>>> >> I use: 
>>> > sqlalchemy.__version__ 
>>> >> '1.0.19' 
>>> >> 
>>> >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King 
>>> написал: 
>>> >>> 
>>> >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан  
>>> wrote: 
>>> >>> > Hi. 
>>> >>> > I use my own `RoutingSession` and `RoutingQuery` implementation, 
>>> most of 
>>> >>> > it 
>>> >>> > inspired by `sqlalchemy.ext.horizontal_shard`: 
>>> >>> > 
>>> >>> > class RoutingSession(Session): 
>>> >>> > def get_bind(self, mapper=None, clause=None, shard_id=None, 
>>> >>> > **kwargs): 
>>> >>> > original_bind = None 
>>> >>> > try: 
>>> >>> > original_bind = super(RoutingSession, 
>>> self).get_bind(mapper, 
>>> >>> > clause) 
>>> >>> > except UnboundExecutionError: 
>>> >>> > # may not be bound 
>>> >>> > pass 
>>> >>> > if shard_id is None: 
>>> >>> > shard_id = TenantIDStorage.get_shard_id()  # just 
>>> global 
>>> >>> > storage 
>>> >>> > bind_for_shard = self.__binds[shard_id] 
>>> >>> > if original_bind is not None and original_bind.url == 
>>> >>> > bind_for_shard.url: 
>>> >>> > return original_bind 
>>> >>> > else: 
>>> >>> > return bind_for_shard 
>>> >>> > 
>>> >>> > def __init__(self, shards=None, query_cls=CachingQuery, 
>>> >>> > engines_factory=None, **kwargs): 
>>> >>> > super(RoutingSession, self).__init__(query_cls=query_cls, 
>>> >>> > **kwargs) 
>>> >>> > self.__binds = {} 
>>> >>> > self.engines_factory = engines_factory 
>>> >>> > if shards is not None: 
>>> >>> > self.update_shards(**shards) 
>>> >>> > 
>>> >>> > def _add_bind(self, key, bind): 
>>> >>> > self.__binds[key] = bind 
>>> >>> > 
>>> >>> > 
>>> >>> > 
>>> >>> > 
>>> >>> > class RoutingQuery(Query): 
>>> >>> > def __init__(self, *args, **kwargs): 
>>> >>> > super(RoutingQuery, self).__init__(*args, **kwargs) 
>>> >>> > self._shard_id = 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

I see that it is not happened when "bind" passed directly to "sessionmaker"

сб, 21 окт. 2017 г. в 18:33, Антонио Антуан :

>
>
> пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer
> написал:
>
>> On Fri, Oct 20, 2017 at 11:05 AM, Simon King 
>> wrote:
>> > The "is not None" is important when checking a variable that may
>> > contain a ClauseElement, precisely because ClauseElement defines that
>> > __bool__ method.
>> >
>> > However, in Session.get_bind(), "mapper" is not supposed to contain a
>> > ClauseElement. It should either be an instance of
>> > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a
>> > valid and concise way to write it. Comparing to None might be strictly
>> > more accurate, but it shouldn't be necessary.
>>
>> agree, "mapper" means "mapper" and it is not supposed to be a
>> ClauseElement.   This sounds like arguments are not being passed
>> correctly somewhere.   Would need a demonstration script if something
>> in SQLAlchemy itself is claimed to be making the mistake.
>>
>
> Ok, that code produces mentioned error:
> https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
> Traceback (most recent call last):
>   File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in 
> Session.query(with_recursive).set_shard('default').all()
>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2654,
> in all
>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2802,
> in __iter__
>   File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in
> _execute_and_instances
> shard_id=self._shard_id).execute(
>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2806,
> in _connection_from_session
>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 984,
> in connection
>   File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in get_bind
> original_bind = super(RoutingSession, self).get_bind(mapper, clause)
>   File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line
> 1336, in get_bind
>   File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line
> 539, in __bool__
> TypeError: Boolean value of this clause is not defined
>
>
>> >
>> > Simon
>>
> >
>> >
>> >
>> > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан 
>> wrote:
>> >> Really, `mapper` contains this: > >> 0x7fe673fb0a50; group_getter>. But does it changes something? I
>> already
>> >> encountered this problem in my code, when I checked sqla-objects
>> existance
>> >> without "is not None", project was broken. Is it normal to omit that
>> >> condition in sqlalchemy code?
>> >>
>> >> I use:
>> > sqlalchemy.__version__
>> >> '1.0.19'
>> >>
>> >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King
>> написал:
>> >>>
>> >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан 
>> wrote:
>> >>> > Hi.
>> >>> > I use my own `RoutingSession` and `RoutingQuery` implementation,
>> most of
>> >>> > it
>> >>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>> >>> >
>> >>> > class RoutingSession(Session):
>> >>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
>> >>> > **kwargs):
>> >>> > original_bind = None
>> >>> > try:
>> >>> > original_bind = super(RoutingSession,
>> self).get_bind(mapper,
>> >>> > clause)
>> >>> > except UnboundExecutionError:
>> >>> > # may not be bound
>> >>> > pass
>> >>> > if shard_id is None:
>> >>> > shard_id = TenantIDStorage.get_shard_id()  # just
>> global
>> >>> > storage
>> >>> > bind_for_shard = self.__binds[shard_id]
>> >>> > if original_bind is not None and original_bind.url ==
>> >>> > bind_for_shard.url:
>> >>> > return original_bind
>> >>> > else:
>> >>> > return bind_for_shard
>> >>> >
>> >>> > def __init__(self, shards=None, query_cls=CachingQuery,
>> >>> > engines_factory=None, **kwargs):
>> >>> > super(RoutingSession, self).__init__(query_cls=query_cls,
>> >>> > **kwargs)
>> >>> > self.__binds = {}
>> >>> > self.engines_factory = engines_factory
>> >>> > if shards is not None:
>> >>> > self.update_shards(**shards)
>> >>> >
>> >>> > def _add_bind(self, key, bind):
>> >>> > self.__binds[key] = bind
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > class RoutingQuery(Query):
>> >>> > def __init__(self, *args, **kwargs):
>> >>> > super(RoutingQuery, self).__init__(*args, **kwargs)
>> >>> > self._shard_id = TenantIDStorage.get_shard_id()
>> >>> >
>> >>> > def get(self, ident):
>> >>> > self._check_bound_to_shard()
>> >>> > return super(CachingQuery, self).get(ident)
>> >>> >
>> >>> > def _check_bound_to_shard(self):
>> >>> > if self._shard_id is None:
>> >>> > raise ValueError('query not bound to any 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

пятница, 20 октября 2017 г., 20:50:52 UTC+3 пользователь Mike Bayer написал:
>
> On Fri, Oct 20, 2017 at 11:05 AM, Simon King  > wrote: 
> > The "is not None" is important when checking a variable that may 
> > contain a ClauseElement, precisely because ClauseElement defines that 
> > __bool__ method. 
> > 
> > However, in Session.get_bind(), "mapper" is not supposed to contain a 
> > ClauseElement. It should either be an instance of 
> > sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a 
> > valid and concise way to write it. Comparing to None might be strictly 
> > more accurate, but it shouldn't be necessary. 
>
> agree, "mapper" means "mapper" and it is not supposed to be a 
> ClauseElement.   This sounds like arguments are not being passed 
> correctly somewhere.   Would need a demonstration script if something 
> in SQLAlchemy itself is claimed to be making the mistake. 
>

Ok, that code produces mentioned 
error: https://gist.github.com/aCLr/113ac292c05bdb01e964d8e9884d6e5f
Traceback (most recent call last):
  File "/home/anton/Projects/proj/core/run/stuff.py", line 100, in 
Session.query(with_recursive).set_shard('default').all()
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2654, 
in all
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2802, 
in __iter__
  File "/home/anton/Projects/proj/core/run/stuff.py", line 28, in 
_execute_and_instances
shard_id=self._shard_id).execute(
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 2806, 
in _connection_from_session
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 984, 
in connection
  File "/home/anton/Projects/proj/core/run/stuff.py", line 40, in get_bind
original_bind = super(RoutingSession, self).get_bind(mapper, clause)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 1336, 
in get_bind
  File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line 539, 
in __bool__
TypeError: Boolean value of this clause is not defined
 

> > 
> > Simon 
> > 
> > 
> > 
> > On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан  > wrote: 
> >> Really, `mapper` contains this:  >> 0x7fe673fb0a50; group_getter>. But does it changes something? I already 
> >> encountered this problem in my code, when I checked sqla-objects 
> existance 
> >> without "is not None", project was broken. Is it normal to omit that 
> >> condition in sqlalchemy code? 
> >> 
> >> I use: 
> > sqlalchemy.__version__ 
> >> '1.0.19' 
> >> 
> >> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King 
> написал: 
> >>> 
> >>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан  
> wrote: 
> >>> > Hi. 
> >>> > I use my own `RoutingSession` and `RoutingQuery` implementation, 
> most of 
> >>> > it 
> >>> > inspired by `sqlalchemy.ext.horizontal_shard`: 
> >>> > 
> >>> > class RoutingSession(Session): 
> >>> > def get_bind(self, mapper=None, clause=None, shard_id=None, 
> >>> > **kwargs): 
> >>> > original_bind = None 
> >>> > try: 
> >>> > original_bind = super(RoutingSession, 
> self).get_bind(mapper, 
> >>> > clause) 
> >>> > except UnboundExecutionError: 
> >>> > # may not be bound 
> >>> > pass 
> >>> > if shard_id is None: 
> >>> > shard_id = TenantIDStorage.get_shard_id()  # just global 
> >>> > storage 
> >>> > bind_for_shard = self.__binds[shard_id] 
> >>> > if original_bind is not None and original_bind.url == 
> >>> > bind_for_shard.url: 
> >>> > return original_bind 
> >>> > else: 
> >>> > return bind_for_shard 
> >>> > 
> >>> > def __init__(self, shards=None, query_cls=CachingQuery, 
> >>> > engines_factory=None, **kwargs): 
> >>> > super(RoutingSession, self).__init__(query_cls=query_cls, 
> >>> > **kwargs) 
> >>> > self.__binds = {} 
> >>> > self.engines_factory = engines_factory 
> >>> > if shards is not None: 
> >>> > self.update_shards(**shards) 
> >>> > 
> >>> > def _add_bind(self, key, bind): 
> >>> > self.__binds[key] = bind 
> >>> > 
> >>> > 
> >>> > 
> >>> > 
> >>> > class RoutingQuery(Query): 
> >>> > def __init__(self, *args, **kwargs): 
> >>> > super(RoutingQuery, self).__init__(*args, **kwargs) 
> >>> > self._shard_id = TenantIDStorage.get_shard_id() 
> >>> > 
> >>> > def get(self, ident): 
> >>> > self._check_bound_to_shard() 
> >>> > return super(CachingQuery, self).get(ident) 
> >>> > 
> >>> > def _check_bound_to_shard(self): 
> >>> > if self._shard_id is None: 
> >>> > raise ValueError('query not bound to any shard') 
> >>> > 
> >>> > def _execute_and_instances(self, querycontext): 
> >>> > self._check_bound_to_shard() 
> >>> > querycontext.attributes['shard_id'] = self._shard_id 
> >>> > 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Mike Bayer]

On Fri, Oct 20, 2017 at 11:05 AM, Simon King  wrote:
> The "is not None" is important when checking a variable that may
> contain a ClauseElement, precisely because ClauseElement defines that
> __bool__ method.
>
> However, in Session.get_bind(), "mapper" is not supposed to contain a
> ClauseElement. It should either be an instance of
> sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a
> valid and concise way to write it. Comparing to None might be strictly
> more accurate, but it shouldn't be necessary.

agree, "mapper" means "mapper" and it is not supposed to be a
ClauseElement.   This sounds like arguments are not being passed
correctly somewhere.   Would need a demonstration script if something
in SQLAlchemy itself is claimed to be making the mistake.



>
> Simon
>
>
>
> On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан  wrote:
>> Really, `mapper` contains this: > 0x7fe673fb0a50; group_getter>. But does it changes something? I already
>> encountered this problem in my code, when I checked sqla-objects existance
>> without "is not None", project was broken. Is it normal to omit that
>> condition in sqlalchemy code?
>>
>> I use:
> sqlalchemy.__version__
>> '1.0.19'
>>
>> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King написал:
>>>
>>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан  wrote:
>>> > Hi.
>>> > I use my own `RoutingSession` and `RoutingQuery` implementation, most of
>>> > it
>>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>>> >
>>> > class RoutingSession(Session):
>>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
>>> > **kwargs):
>>> > original_bind = None
>>> > try:
>>> > original_bind = super(RoutingSession, self).get_bind(mapper,
>>> > clause)
>>> > except UnboundExecutionError:
>>> > # may not be bound
>>> > pass
>>> > if shard_id is None:
>>> > shard_id = TenantIDStorage.get_shard_id()  # just global
>>> > storage
>>> > bind_for_shard = self.__binds[shard_id]
>>> > if original_bind is not None and original_bind.url ==
>>> > bind_for_shard.url:
>>> > return original_bind
>>> > else:
>>> > return bind_for_shard
>>> >
>>> > def __init__(self, shards=None, query_cls=CachingQuery,
>>> > engines_factory=None, **kwargs):
>>> > super(RoutingSession, self).__init__(query_cls=query_cls,
>>> > **kwargs)
>>> > self.__binds = {}
>>> > self.engines_factory = engines_factory
>>> > if shards is not None:
>>> > self.update_shards(**shards)
>>> >
>>> > def _add_bind(self, key, bind):
>>> > self.__binds[key] = bind
>>> >
>>> >
>>> >
>>> >
>>> > class RoutingQuery(Query):
>>> > def __init__(self, *args, **kwargs):
>>> > super(RoutingQuery, self).__init__(*args, **kwargs)
>>> > self._shard_id = TenantIDStorage.get_shard_id()
>>> >
>>> > def get(self, ident):
>>> > self._check_bound_to_shard()
>>> > return super(CachingQuery, self).get(ident)
>>> >
>>> > def _check_bound_to_shard(self):
>>> > if self._shard_id is None:
>>> > raise ValueError('query not bound to any shard')
>>> >
>>> > def _execute_and_instances(self, querycontext):
>>> > self._check_bound_to_shard()
>>> > querycontext.attributes['shard_id'] = self._shard_id
>>> > result = self._connection_from_session(
>>> > mapper=self._mapper_zero(),
>>> > shard_id=self._shard_id).execute(
>>> > querycontext.statement,
>>> > self._params
>>> > )
>>> > return self.instances(result, querycontext)
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Sometimes I got this error:
>>> >
>>> >   File "/home/anton/Projects/proj/admin/proj/admin/views/stats.py", line
>>> > 898, in _get_filters_from_request
>>> > control_groups = [g.id for g in
>>> > User.get_own_groups_query(current_user.id)]
>>> >   File
>>> >
>>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
>>> > line 2802, in __iter__
>>> > return self._execute_and_instances(context)
>>> >   File
>>> >
>>> > "/home/anton/Projects/proj/core/proj/core/orm_extensions/rouing_session.py",
>>> > line 105, in _execute_and_instances
>>> > shard_id=self._shard_id).execute(
>>> >   File
>>> >
>>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
>>> > line 2806, in _connection_from_session
>>> > **kw)
>>> >   File
>>> >
>>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
>>> > line 984, in connection
>>> > bind = self.get_bind(mapper, clause=clause, **kw)
>>> >   File
>>> >
>>> > "/home/anton/Projects/proj/core/proj/core/orm_extensions/cachingquery.py",
>>> > line 279, in get_bind
>>> > original_bind = 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Simon King]

The "is not None" is important when checking a variable that may
contain a ClauseElement, precisely because ClauseElement defines that
__bool__ method.

However, in Session.get_bind(), "mapper" is not supposed to contain a
ClauseElement. It should either be an instance of
sqlalchemy.orm.mapper.Mapper, or None, in which case "if mapper:" is a
valid and concise way to write it. Comparing to None might be strictly
more accurate, but it shouldn't be necessary.

Simon



On Fri, Oct 20, 2017 at 3:17 PM, Антонио Антуан  wrote:
> Really, `mapper` contains this:  0x7fe673fb0a50; group_getter>. But does it changes something? I already
> encountered this problem in my code, when I checked sqla-objects existance
> without "is not None", project was broken. Is it normal to omit that
> condition in sqlalchemy code?
>
> I use:
 sqlalchemy.__version__
> '1.0.19'
>
> пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King написал:
>>
>> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан  wrote:
>> > Hi.
>> > I use my own `RoutingSession` and `RoutingQuery` implementation, most of
>> > it
>> > inspired by `sqlalchemy.ext.horizontal_shard`:
>> >
>> > class RoutingSession(Session):
>> > def get_bind(self, mapper=None, clause=None, shard_id=None,
>> > **kwargs):
>> > original_bind = None
>> > try:
>> > original_bind = super(RoutingSession, self).get_bind(mapper,
>> > clause)
>> > except UnboundExecutionError:
>> > # may not be bound
>> > pass
>> > if shard_id is None:
>> > shard_id = TenantIDStorage.get_shard_id()  # just global
>> > storage
>> > bind_for_shard = self.__binds[shard_id]
>> > if original_bind is not None and original_bind.url ==
>> > bind_for_shard.url:
>> > return original_bind
>> > else:
>> > return bind_for_shard
>> >
>> > def __init__(self, shards=None, query_cls=CachingQuery,
>> > engines_factory=None, **kwargs):
>> > super(RoutingSession, self).__init__(query_cls=query_cls,
>> > **kwargs)
>> > self.__binds = {}
>> > self.engines_factory = engines_factory
>> > if shards is not None:
>> > self.update_shards(**shards)
>> >
>> > def _add_bind(self, key, bind):
>> > self.__binds[key] = bind
>> >
>> >
>> >
>> >
>> > class RoutingQuery(Query):
>> > def __init__(self, *args, **kwargs):
>> > super(RoutingQuery, self).__init__(*args, **kwargs)
>> > self._shard_id = TenantIDStorage.get_shard_id()
>> >
>> > def get(self, ident):
>> > self._check_bound_to_shard()
>> > return super(CachingQuery, self).get(ident)
>> >
>> > def _check_bound_to_shard(self):
>> > if self._shard_id is None:
>> > raise ValueError('query not bound to any shard')
>> >
>> > def _execute_and_instances(self, querycontext):
>> > self._check_bound_to_shard()
>> > querycontext.attributes['shard_id'] = self._shard_id
>> > result = self._connection_from_session(
>> > mapper=self._mapper_zero(),
>> > shard_id=self._shard_id).execute(
>> > querycontext.statement,
>> > self._params
>> > )
>> > return self.instances(result, querycontext)
>> >
>> >
>> >
>> >
>> >
>> >
>> > Sometimes I got this error:
>> >
>> >   File "/home/anton/Projects/proj/admin/proj/admin/views/stats.py", line
>> > 898, in _get_filters_from_request
>> > control_groups = [g.id for g in
>> > User.get_own_groups_query(current_user.id)]
>> >   File
>> >
>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
>> > line 2802, in __iter__
>> > return self._execute_and_instances(context)
>> >   File
>> >
>> > "/home/anton/Projects/proj/core/proj/core/orm_extensions/rouing_session.py",
>> > line 105, in _execute_and_instances
>> > shard_id=self._shard_id).execute(
>> >   File
>> >
>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
>> > line 2806, in _connection_from_session
>> > **kw)
>> >   File
>> >
>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
>> > line 984, in connection
>> > bind = self.get_bind(mapper, clause=clause, **kw)
>> >   File
>> >
>> > "/home/anton/Projects/proj/core/proj/core/orm_extensions/cachingquery.py",
>> > line 279, in get_bind
>> > original_bind = super(RoutingSession, self).get_bind(mapper, clause)
>> >   File
>> >
>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
>> > line 1336, in get_bind
>> > if mapper and mapper.mapped_table.bind:
>> >   File
>> >
>> > "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
>> > line 539, in __bool__
>> > raise TypeError("Boolean value of this clause is not defined")
>> > TypeError: Boolean value of this 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

Really, `mapper` contains this: . But does it changes something? I already 
encountered this problem in my code, when I checked sqla-objects existance 
without "is not None", project was broken. Is it normal to omit that 
condition in sqlalchemy code?

I use:
>>> sqlalchemy.__version__
'1.0.19'

пятница, 20 октября 2017 г., 16:42:23 UTC+3 пользователь Simon King написал:
>
> On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан  > wrote: 
> > Hi. 
> > I use my own `RoutingSession` and `RoutingQuery` implementation, most of 
> it 
> > inspired by `sqlalchemy.ext.horizontal_shard`: 
> > 
> > class RoutingSession(Session): 
> > def get_bind(self, mapper=None, clause=None, shard_id=None, 
> **kwargs): 
> > original_bind = None 
> > try: 
> > original_bind = super(RoutingSession, self).get_bind(mapper, 
> > clause) 
> > except UnboundExecutionError: 
> > # may not be bound 
> > pass 
> > if shard_id is None: 
> > shard_id = TenantIDStorage.get_shard_id()  # just global 
> storage 
> > bind_for_shard = self.__binds[shard_id] 
> > if original_bind is not None and original_bind.url == 
> > bind_for_shard.url: 
> > return original_bind 
> > else: 
> > return bind_for_shard 
> > 
> > def __init__(self, shards=None, query_cls=CachingQuery, 
> > engines_factory=None, **kwargs): 
> > super(RoutingSession, self).__init__(query_cls=query_cls, 
> **kwargs) 
> > self.__binds = {} 
> > self.engines_factory = engines_factory 
> > if shards is not None: 
> > self.update_shards(**shards) 
> > 
> > def _add_bind(self, key, bind): 
> > self.__binds[key] = bind 
> > 
> > 
> > 
> > 
> > class RoutingQuery(Query): 
> > def __init__(self, *args, **kwargs): 
> > super(RoutingQuery, self).__init__(*args, **kwargs) 
> > self._shard_id = TenantIDStorage.get_shard_id() 
> > 
> > def get(self, ident): 
> > self._check_bound_to_shard() 
> > return super(CachingQuery, self).get(ident) 
> > 
> > def _check_bound_to_shard(self): 
> > if self._shard_id is None: 
> > raise ValueError('query not bound to any shard') 
> > 
> > def _execute_and_instances(self, querycontext): 
> > self._check_bound_to_shard() 
> > querycontext.attributes['shard_id'] = self._shard_id 
> > result = self._connection_from_session( 
> > mapper=self._mapper_zero(), 
> > shard_id=self._shard_id).execute( 
> > querycontext.statement, 
> > self._params 
> > ) 
> > return self.instances(result, querycontext) 
> > 
> > 
> > 
> > 
> > 
> > 
> > Sometimes I got this error: 
> > 
> >   File "/home/anton/Projects/proj/admin/proj/admin/views/stats.py", line 
> > 898, in _get_filters_from_request 
> > control_groups = [g.id for g in 
> > User.get_own_groups_query(current_user.id)] 
> >   File 
> > 
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
>  
>
> > line 2802, in __iter__ 
> > return self._execute_and_instances(context) 
> >   File 
> > 
> "/home/anton/Projects/proj/core/proj/core/orm_extensions/rouing_session.py", 
>
> > line 105, in _execute_and_instances 
> > shard_id=self._shard_id).execute( 
> >   File 
> > 
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
>  
>
> > line 2806, in _connection_from_session 
> > **kw) 
> >   File 
> > 
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
>  
>
> > line 984, in connection 
> > bind = self.get_bind(mapper, clause=clause, **kw) 
> >   File 
> > 
> "/home/anton/Projects/proj/core/proj/core/orm_extensions/cachingquery.py", 
> > line 279, in get_bind 
> > original_bind = super(RoutingSession, self).get_bind(mapper, clause) 
> >   File 
> > 
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
>  
>
> > line 1336, in get_bind 
> > if mapper and mapper.mapped_table.bind: 
> >   File 
> > 
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
>  
>
> > line 539, in __bool__ 
> > raise TypeError("Boolean value of this clause is not defined") 
> > TypeError: Boolean value of this clause is not defined 
> > 
> > 
> > 
> > 
> > I found that `mapper` in original `get_bind` always checks with `is not 
> > None` condition. There is only one place where condition omitted: `if 
> mapper 
> > and mapper.mapped_table.bind...`. 
> > Possibly it is not correct? Or my code does not do something important? 
>
> Are you able to catch this error in a debugger? 
>
> I'm going to make a wild guess that "mapper" at this point is some 
> sort of SQL construct (such as a column object or mapped property), 
> rather than an actual mapper. 
>
> You don't say which version of 

Re: [sqlalchemy] mapper existance checks possibly wrong

[Simon King]

On Fri, Oct 20, 2017 at 2:15 PM, Антонио Антуан  wrote:
> Hi.
> I use my own `RoutingSession` and `RoutingQuery` implementation, most of it
> inspired by `sqlalchemy.ext.horizontal_shard`:
>
> class RoutingSession(Session):
> def get_bind(self, mapper=None, clause=None, shard_id=None, **kwargs):
> original_bind = None
> try:
> original_bind = super(RoutingSession, self).get_bind(mapper,
> clause)
> except UnboundExecutionError:
> # may not be bound
> pass
> if shard_id is None:
> shard_id = TenantIDStorage.get_shard_id()  # just global storage
> bind_for_shard = self.__binds[shard_id]
> if original_bind is not None and original_bind.url ==
> bind_for_shard.url:
> return original_bind
> else:
> return bind_for_shard
>
> def __init__(self, shards=None, query_cls=CachingQuery,
> engines_factory=None, **kwargs):
> super(RoutingSession, self).__init__(query_cls=query_cls, **kwargs)
> self.__binds = {}
> self.engines_factory = engines_factory
> if shards is not None:
> self.update_shards(**shards)
>
> def _add_bind(self, key, bind):
> self.__binds[key] = bind
>
>
>
>
> class RoutingQuery(Query):
> def __init__(self, *args, **kwargs):
> super(RoutingQuery, self).__init__(*args, **kwargs)
> self._shard_id = TenantIDStorage.get_shard_id()
>
> def get(self, ident):
> self._check_bound_to_shard()
> return super(CachingQuery, self).get(ident)
>
> def _check_bound_to_shard(self):
> if self._shard_id is None:
> raise ValueError('query not bound to any shard')
>
> def _execute_and_instances(self, querycontext):
> self._check_bound_to_shard()
> querycontext.attributes['shard_id'] = self._shard_id
> result = self._connection_from_session(
> mapper=self._mapper_zero(),
> shard_id=self._shard_id).execute(
> querycontext.statement,
> self._params
> )
> return self.instances(result, querycontext)
>
>
>
>
>
>
> Sometimes I got this error:
>
>   File "/home/anton/Projects/proj/admin/proj/admin/views/stats.py", line
> 898, in _get_filters_from_request
> control_groups = [g.id for g in
> User.get_own_groups_query(current_user.id)]
>   File
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
> line 2802, in __iter__
> return self._execute_and_instances(context)
>   File
> "/home/anton/Projects/proj/core/proj/core/orm_extensions/rouing_session.py",
> line 105, in _execute_and_instances
> shard_id=self._shard_id).execute(
>   File
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
> line 2806, in _connection_from_session
> **kw)
>   File
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 984, in connection
> bind = self.get_bind(mapper, clause=clause, **kw)
>   File
> "/home/anton/Projects/proj/core/proj/core/orm_extensions/cachingquery.py",
> line 279, in get_bind
> original_bind = super(RoutingSession, self).get_bind(mapper, clause)
>   File
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
> line 1336, in get_bind
> if mapper and mapper.mapped_table.bind:
>   File
> "/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
> line 539, in __bool__
> raise TypeError("Boolean value of this clause is not defined")
> TypeError: Boolean value of this clause is not defined
>
>
>
>
> I found that `mapper` in original `get_bind` always checks with `is not
> None` condition. There is only one place where condition omitted: `if mapper
> and mapper.mapped_table.bind...`.
> Possibly it is not correct? Or my code does not do something important?

Are you able to catch this error in a debugger?

I'm going to make a wild guess that "mapper" at this point is some
sort of SQL construct (such as a column object or mapped property),
rather than an actual mapper.

You don't say which version of SQLAlchemy you are using, but since
you've overridden _execute_and_instances, I wonder if you need to use
something like query._bind_mapper() as used by _get_bind_args:



Hope that helps,

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an 

[sqlalchemy] mapper existance checks possibly wrong

[Антонио Антуан]

Hi.
I use my own `RoutingSession` and `RoutingQuery` implementation, most of it 
inspired by `sqlalchemy.ext.horizontal_shard`:

class RoutingSession(Session):
def get_bind(self, mapper=None, clause=None, shard_id=None, **kwargs):
original_bind = None
try:
original_bind = super(RoutingSession, self).get_bind(mapper, clause)
except UnboundExecutionError:
# may not be bound
pass
if shard_id is None:
shard_id = TenantIDStorage.get_shard_id()  # just global storage
bind_for_shard = self.__binds[shard_id]
if original_bind is not None and original_bind.url == 
bind_for_shard.url:
return original_bind
else:
return bind_for_shard

def __init__(self, shards=None, query_cls=CachingQuery, 
engines_factory=None, **kwargs):
super(RoutingSession, self).__init__(query_cls=query_cls, **kwargs)
self.__binds = {}
self.engines_factory = engines_factory
if shards is not None:
self.update_shards(**shards)

def _add_bind(self, key, bind):
self.__binds[key] = bind




class RoutingQuery(Query):
def __init__(self, *args, **kwargs):
super(RoutingQuery, self).__init__(*args, **kwargs)
self._shard_id = TenantIDStorage.get_shard_id()

def get(self, ident):
self._check_bound_to_shard()
return super(CachingQuery, self).get(ident)

def _check_bound_to_shard(self):
if self._shard_id is None:
raise ValueError('query not bound to any shard')

def _execute_and_instances(self, querycontext):
self._check_bound_to_shard()
querycontext.attributes['shard_id'] = self._shard_id
result = self._connection_from_session(
mapper=self._mapper_zero(),
shard_id=self._shard_id).execute(
querycontext.statement,
self._params
)
return self.instances(result, querycontext)






Sometimes I got this error:

  File "/home/anton/Projects/proj/admin/proj/admin/views/stats.py", line 898, 
in _get_filters_from_request
control_groups = [g.id for g in User.get_own_groups_query(current_user.id)]
  File 
"/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
 line 2802, in __iter__
return self._execute_and_instances(context)
  File 
"/home/anton/Projects/proj/core/proj/core/orm_extensions/rouing_session.py", 
line 105, in _execute_and_instances
shard_id=self._shard_id).execute(
  File 
"/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
 line 2806, in _connection_from_session
**kw)
  File 
"/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
 line 984, in connection
bind = self.get_bind(mapper, clause=clause, **kw)
  File 
"/home/anton/Projects/proj/core/proj/core/orm_extensions/cachingquery.py", line 
279, in get_bind
original_bind = super(RoutingSession, self).get_bind(mapper, clause)
  File 
"/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/orm/session.py",
 line 1336, in get_bind
if mapper and mapper.mapped_table.bind:
  File 
"/home/anton/Projects/proj/.venv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
 line 539, in __bool__
raise TypeError("Boolean value of this clause is not defined")
TypeError: Boolean value of this clause is not defined




I found that `mapper` in original `get_bind` always checks with `is not 
None` condition. There is only one place where condition omitted: `if 
mapper and mapper.mapped_table.bind...`. 
Possibly it is not correct? Or my code does not do something important? 

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.