Sorry for top post: thanks to Tim Burke, we are past this and a few other problems. Here's the latest work:
Swift - Review from Tim : https://review.openstack.org/#/c/401397/ Swift - My review (WIP) : https://review.openstack.org/#/c/416084/ DevStack - My review : https://review.openstack.org/#/c/416064/ With all these, we get to the point where s-proxy gets "stuck" and the tls-proxy times out, logs are here: http://logs.openstack.org/00/412500/9/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/d453c3f/logs/ Happy to turn this over to the swift team from here on. One more good news, Thanks to the Keystone team for their very first DevStack+python3.5 based functional test (gate-keystone-dsvm-py35-functional-v3-only-ubuntu-xenial-nv) with just a couple of failures left to go: https://review.openstack.org/#/c/412500/ Thanks, Dims On Wed, Jan 4, 2017 at 4:49 PM, Doug Hellmann <d...@doughellmann.com> wrote: > Excerpts from John Dickinson's message of 2017-01-03 12:00:07 -0800: >> >> On 3 Jan 2017, at 10:38, Doug Hellmann wrote: >> >> > Excerpts from John Dickinson's message of 2017-01-03 09:02:19 -0800: >> >> >> >> On 2 Jan 2017, at 13:06, Davanum Srinivas wrote: >> >> >> >>> Folks, >> >>> >> >>> Short Story : >> >>> [1] has merged in devstack, it adds support for a python 3.5 based >> >>> up/down devstack test that just starts services and brings them down. >> >>> see [2] for a test run. >> >>> >> >>> Need help from swift folks: >> >>> Swift still needs work i have gotten as far as [3] UnpicklingError on >> >>> ring data using [4][5][6][7]. Can someone from the swift team pick >> >>> this up? >> >>> Once you get this working, please add "swift" to the white list in [8] >> >>> and remove the disable_service for swift services in [9] >> >> >> >> IIRC the issue is the differences between pickle, json, and arrays in py2 >> >> vs py3 (short summary: you can't deserialize in py3 stuff that was >> >> serialized in py2 without first changing the py2 code). >> > >> > Is that right? It seems like it would be the other way around. There >> > are newer pickle protocols in 3 that aren't available at all for 2. >> > >> > There are also some new options to the load() function in 3 to do >> > things like fix imports for standard library modules that were >> > renamed and set the right default string encoding to make it possible >> > to change the *3* code to be able to more easily load a pickle >> > created by 2 [1]. Is that what you meant? >> >> Nah, it's not the pickle protocol. It's the different way (some versions of) >> py27 [de]serializes arrays vs how py3 does it. The following breaks for >> py2.7.10 and works for py2.7.12 (.11 is untested). >> >> python3 -c 'import array, pickle, os, sys; pickle.dump(array.array("I", [0, >> 0, 0]), os.fdopen(1, "wb"), protocol=2)' | python -c 'import pickle, os, >> sys; print(pickle.load(os.fdopen(0, "rb")))' > > o_O > > Sigh. > >> >> So maybe there are ways to always ensure doing the right thing through some >> combination of try/excepts, "if six.PY3" blocks, plus lots of docs for ops >> to make sure upgrades go smoothly, but the real solution is to not use >> pickle. This is doubly true when considering that the data structure will be >> used by non-python code, too. > > "Friends don't let their friends use pickle for portable persistence." > >> >> This is one of the things to put on the list for py3, and it certainly is >> not the last. > > It would be good to start building up the list in the "Common Pitfalls" > section of https://wiki.openstack.org/wiki/Python3 > > Doug > >> >> --John >> >> > >> > Doug >> > >> > [1] https://docs.python.org/3.5/library/pickle.html#pickle.load >> > >> >> >> >> We're tracking this at https://bugs.launchpad.net/swift/+bug/1644387 and >> >> have a patch at https://review.openstack.org/#/c/401397/, but I can't >> >> guarantee that services will start as soon as that patch lands. (i.e. >> >> it's necessary, but might not be sufficient) >> >> >> >> --John >> >> >> >>> >> >>> Other teams: >> >>> Please consider adding DSVM jobs with USE_PYTHON3=True for your >> >>> projects. This will hopefully help us get to our Pike goal for Python >> >>> 3.x [10] >> >>> >> >>> Please stop by #openstack-python3 channel to chat. >> >>> >> >>> Thanks, >> >>> Dims >> >>> >> >>> [1] https://review.openstack.org/#/c/414176/ >> >>> [2] >> >>> http://logs.openstack.org/76/414176/11/check/gate-devstack-dsvm-py35-updown-ubuntu-xenial-nv/ >> >>> [3] >> >>> http://logs.openstack.org/00/412500/7/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/f5a7fe7/logs/screen-s-proxy.txt.gz >> >>> [4] https://review.openstack.org/#/c/412500/ >> >>> [5] https://review.openstack.org/#/c/414727/ >> >>> [6] https://review.openstack.org/#/c/416064/ >> >>> [7] https://review.openstack.org/#/c/416084/ >> >>> [8] https://review.openstack.org/#/c/414176/11/inc/python >> >>> [9] >> >>> https://review.openstack.org/#/c/413775/5/jenkins/jobs/devstack-gate.yaml >> >>> [10] https://review.openstack.org/#/c/349069/ >> >>> >> >>> -- >> >>> Davanum Srinivas :: https://twitter.com/dims >> >>> >> >>> __________________________________________________________________________ >> >>> OpenStack Development Mailing List (not for usage questions) >> >>> Unsubscribe: >> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> > __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
