that was actually quite easy! https://github.com/sqlalchemy/sqlalchemy/issues/4507
On Tue, Feb 19, 2019 at 7:18 PM Mike Bayer <mike...@zzzcomputing.com> wrote: > > this will be moving to a github issue, and also I will attempt to > create a proof of concept. > > Everything that happens inside of _memoized_attr__simple_lazy_clause > involves creating copies of things and the state of the loader > strategy is not modified until it's done. There is one "unique=False" > flag set on existing bindparam objects, however if multiple threads > all did that at the same time, it doesn't change the outcome. The > two elements of the tuple returned, "crtierion" and "params", need to > match, however these are returned as a tuple and that's what's > assigned; if two different threads both ran the method at the same > time, one would replace the other....but I guess the baked query is > getting the state of the earlier one cached in some way, but, I don't > see how the two generations are meaningfully different....but that's > where something would need to be failing. Best option would be if > it's not a problem for this code to run concurrently, however we can > use the mapper._CONFIGURE_MUTEX for this, but there are probably a lot > of other places where this kind of thing can happen if the > introduction of baked + memoized attrs has this issue. > > On Tue, Feb 19, 2019 at 4:03 PM Krzysztof Sulejczak > <krzysztof.sulejc...@gmail.com> wrote: > > > > Hi, > > > > I'm maintaining a multi-threaded application which uses sqlalchemy to > > access mysql db. Some time ago we decided to upgrade > > sqlalchemy version from 1.1.18 to 1.2.14 and since then we started > > observing strange problems, the application started > > throwing IntegrityError in code like that: > > > > translation = item.translation # translation is a relation > > if translation is None: > > session.add(Translation(item=item)) # this throws error > > > > Strange thing was that these errors happened rather rarely, there were a > > few hundred application processes running and only > > a few of them were throwing such errors. Restarting those processed made > > the errors vanish. > > I started debugging faulty processes and noticed that sql query generated > > looked like this: > > > > SELECT <truncated list of columns> > > FROM ebay_translations > > WHERE NULL = ebay_translations.item_id > > > > Further debugging led me to LazyLoader._emit_lazyload and BackedQuery > > thingy where I believe problem lays: > > if a thread switch occurs inside _memoized_attr__simple_lazy_clause it > > could happen that BindParameter keys and > > cached relation query will not match each other. > > > > Since that discovery I've been running the application with patched version > > of sqlalchemy for two months now, > > IntegrityErrors are gone. > > > > Can you please take a look at this patch? Or maybe this issue is known and > > there exists workaround for it although I searched > > the mailing list archive and found nothing. > > > > best regards > > Krzysztof Sulejczak > > > > -- > > SQLAlchemy - > > The Python SQL Toolkit and Object Relational Mapper > > > > http://www.sqlalchemy.org/ > > > > To post example code, please provide an MCVE: Minimal, Complete, and > > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > > description. > > --- > > You received this message because you are subscribed to the Google Groups > > "sqlalchemy" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to sqlalchemy+unsubscr...@googlegroups.com. > > To post to this group, send email to sqlalchemy@googlegroups.com. > > Visit this group at https://groups.google.com/group/sqlalchemy. > > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
