Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-14 Thread Mike Bayer
thanks! On Sun, May 14, 2023, at 6:59 PM, Shane Holcombe wrote: > I've tested all the ways I could previously trigger the issue and still, no > Nones seem to even be produced to be filtered out, that fix definitely seems > to have sorted the issue out > > On Sunday, May 14, 2023 at 7:42:54 AM

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-14 Thread Shane Holcombe
I've tested all the ways I could previously trigger the issue and still, no Nones seem to even be produced to be filtered out, that fix definitely seems to have sorted the issue out On Sunday, May 14, 2023 at 7:42:54 AM UTC+10 Shane Holcombe wrote: > Yes so I'm testing the second solution with

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-13 Thread Shane Holcombe
Yes so I'm testing the second solution with a breakpoint on the area that should filter out those Nones, I wanted to see wether the Nones are being removed or are no longer showing up. So far I haven't seen that code be used, which is great, means the race condition thing must have helped. I

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-13 Thread Mike Bayer
the second solution has not only the removal of where the theoretical "race condition" might happen, but it also removes the "None" entries in any case, so even if the "None" thing is coming from somewhere else, they won't make it to the select() statement at the end. On Sat, May 13, 2023,

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-13 Thread Shane Holcombe
I've tested both solutions, the first one, moving the if c is None: return down a few lines seems to work and the generated SQL still seems to be correct in the instances where that code is triggered. >From some early testing as well the other solution seems to work as well, it's hard to test

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
in the below mentioned issue I've created a patch at https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/4615 that removes a thread-sensitive approach to generating an internal structure, which is where I suspect this might be happening (if anywhere), and then it also adds a guard against

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
I've opened https://github.com/sqlalchemy/sqlalchemy/issues/9777 for this. I have another theory where something might be going wrong earler in the chain. This change would be "simpler" in that it removes something that's complicated, prone to race conditions hence seems a good possibility

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Shane Holcombe
There seems to be a few moving parts to this so it's been hard to track down, I can give that fix a try in a few hours when I'm home. I was worried something like that might lose some columns but I'll try and see what happens. Thanks On Saturday, May 13, 2023 at 12:59:33 PM UTC+10 Mike Bayer

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
well, not sure yet how a None is coming in there that is sporadic, or even at all, but if that's what's happening then we would think this patch would fix the problem, the Q is does the query still produce the correct results: diff --git a/lib/sqlalchemy/orm/strategies.py

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
OK, maybe you are onto something with the theory re: JoinedLoader._generate_row_adapter(). will look at that On Fri, May 12, 2023, at 6:16 PM, Shane Holcombe wrote: > Thanks for linking that github issue, completely missed that one. > > Here's the complete stack trace, sorry for not including

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Shane Holcombe
Thanks for linking that github issue, completely missed that one. Here's the complete stack trace, sorry for not including that at the start Traceback (most recent call last): File "/Users/sfh/env/vita/lib/python3.9/site-packages/werkzeug/serving.py", line 333, in run_wsgi

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
I really need to see: 1. actual models to the degree they illustrate the default loaders set up on relationships 2. query in use 3. **complete** stack trace, put it on a github gist if it's too long. every line, the error, top to bottom. thanks On Fri, May 12, 2023, at 11:25 AM, Mike Bayer

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
On Fri, May 12, 2023, at 1:25 AM, Shane Holcombe wrote: > AttributeError: 'NoneType' object has no attribute '_from_objects' > (I've shortened this from the entire traceback) OK what kind of traceback, is it a recursion overflow kind of traceback ? (e.g. many hundreds of repeated series of

Re: [sqlalchemy] Migration issues - Column value None on select with joins

2023-05-12 Thread Mike Bayer
this has also been reported at https://github.com/sqlalchemy/sqlalchemy/discussions/9666 On Fri, May 12, 2023, at 1:25 AM, Shane Holcombe wrote: > > From the digging around that I've done, it seems to be the ColumnLoader > strategy setup_query method sometimes has a value for 'c' of None