On Mon, Jan 23, 2017 at 5:10 PM, Jean-Paul Calderone < exar...@twistedmatrix.com> wrote:
> On Mon, Jan 23, 2017 at 7:08 PM, Craig Rodrigues <rodr...@crodrigues.org> > wrote: > >> >> >> >> I did some more debugging. The callstack is quite deep. :/ >> I used this command to trigger the error: >> >> trial buildbot.test.unit.test_process_buildrequestdistributor.Tes >> tMaybeStartBuilds.test_slow_db >> >> I found in this function in buildbot's BuildRequestsEndpoint.get() >> function ( https://github.com/buildbot/buildbot/blob/master/master/buil >> dbot/data/buildrequests.py#L146 ) there is this line: >> >> >> >> * defer.returnValue( [(yield self.db2data(br)) for br >> in buildrequests])* >> On Python 2, this line returns: >> an object list >> each list entry is a dictionary of name/value pairs that looks like: >> >> [{'buildrequestid': 10, 'complete': False, 'waited_for': False, >> 'claimed_at': None, 'results': -1, 'claimed': False, 'buildsetid': 11, >> 'complete_at': None, 'submitted_at': datetime.datetime(1970, 1, 2, 12, 6, >> 40, tzinfo=tzutc()), 'builderid': 77, 'claimed_by_masterid': None, >> 'priority': 0}, {'buildrequestid': 11, 'complete': False, 'waited_for': >> False, 'claimed_at': None, 'results': -1, 'claimed': False, 'buildsetid': >> 11, 'complete_at': None, 'submitted_at': datetime.datetime(1970, 1, 2, 13, >> 30, tzinfo=tzutc()), 'builderid': 77, 'claimed_by_masterid': None, >> 'priority': 0}] >> >> On Python 3, this returns: >> an object <generator object BuildRequestsEndpoint.get.<loc >> als>.<listcomp> >> of type <class 'generator'> >> >> > Yep. > > On Python 2, [(yield self.db2data(br)) for br in buildrequests] is a list > comprehension. It will have len(buildrequests) elements and each will be > the value sent back in to the generator via the yield expression. > > On Python 3, the same expression is a list of one element which is a > generator expression. > > This form is much clearer, I think, and behaves as intended on both > versions of Python: > > results = [] > for br in buildrequests: > results.append((yield self.db2data(br))) > defer.returnValue(results) > > > Wow, thanks! That fixed it. I submitted those fixes upstream to buildbot. I'm not so familiar with generator expressions and Deferred's so have been playing around with things to try to understand. I went back to the problem code, and changed: * defer.returnValue( [(yield self.db2data(br)) for br in buildrequests])* to: * defer.returnValue(* * list([**(yield self.db2data(br)) for br in buildrequests])* and ran the code under Python 3 and got a return value of: [ Deferred ] instead of: [ {......} ] where the Deferred.results value was the dictionary. How does Python 2 directly go to returning the dictionary out of the Deferred, while Python 3 returns the Deferred inside the list? self.db2data() returns a Deferred. Thanks. -- Craig
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python