On 2019/10/16 21:12, Johan Corveleyn wrote:
On Wed, Oct 16, 2019 at 1:37 PM Yasuhito FUTATSUKI <futat...@poem.co.jp> wrote:
On 2019/10/16 18:10, Johan Corveleyn wrote:
Python 3 for the build and test process is only supported on *nix, not
on Windows.
[[[
Traceback (most recent call last):
File "win-tests.py", line 134, in <module>
cp.items('options'))
File "build\generator\gen_win_dependencies.py", line 306, in __init__
self.find_libraries(False)
File "build\generator\gen_win_dependencies.py", line 327, in find_libraries
self._find_jdk(show_warnings)
File "build\generator\gen_win_dependencies.py", line 1085, in _find_jdk
vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
File "C:\Python37\lib\re.py", line 183, in search
return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object
]]]
The fix is probably easy, but I'm just noting it here so we don't get
ahead of ourselves.
It seems the change addressing for it on swig-py3 branch is a part of
r1822485, the hunks attached.
Ack.
I've looked over the log on swig-py3 branch and picked up more hunks,
and I've made a patch against trunk@1868582.
Could you please test on trunk on Windows with this patch?
This makes me wonder: should that be fixed specifically on trunk, and
nominated for backport to 1.13, so we can possibly claim basic support
for Python 3 in our build and test processes (in at least one released
version) before the end of this year?
Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
for backport to 1.13, so we can have Python 3 support, including swig
bindings?
I prefer the latter, as one of users :) I want to use
tools/hook-scripts/mailer/mailer.py with Python 3.
Thanks,
--
Yasuhito FUTATSUKI <futat...@poem.co.jp>
Fix issue on test on Windows with Python 3.
(cherry-pick of hunks from r1822485 and its fix r1822486, on branch swig-py3)
* build/generator/gen_base.py
(IncludeDependencyInfo._scan_for_includes): Ensure file data is
properly encoded as UTF8. (r1822485, r1822486)
* build/generator/gen_win.py
(WinGeneratorBase.makeguid): Ensure data input to hashlib is binary
and not Unicode. (r1822485)
* build/generator/gen_win_dependencies.py
(GenDependenciesBase._find_swig,
GenDependenciesBase._find_jdk): Ensure output from subprocess is
properly decoded. (r1822485)
Index: build/generator/gen_base.py
===================================================================
--- build/generator/gen_base.py (revision 1868582)
+++ build/generator/gen_base.py (working copy)
@@ -1273,7 +1273,8 @@
Return a dictionary with included full file names as keys and None as
values."""
hdrs = { }
- for line in fileinput.input(fname):
+
+ for line in fileinput.FileInput(fname,
openhook=fileinput.hook_encoded("utf-8")):
match = self._re_include.match(line)
if not match:
continue
Index: build/generator/gen_win.py
===================================================================
--- build/generator/gen_win.py (revision 1868582)
+++ build/generator/gen_win.py (working copy)
@@ -158,6 +158,13 @@
### implement this from scratch using the algorithms described in
### http://www.webdav.org/specs/draft-leach-uuids-guids-01.txt
+ # Ensure data is in byte representation. If it doesn't have an encode
+ # attribute, assume it is already in the correct form.
+ try:
+ data = data.encode('utf8')
+ except AttributeError:
+ pass
+
myhash = hashlib_md5(data).hexdigest()
guid = ("{%s-%s-%s-%s-%s}" % (myhash[0:8], myhash[8:12],
Index: build/generator/gen_win_dependencies.py
===================================================================
--- build/generator/gen_win_dependencies.py (revision 1868582)
+++ build/generator/gen_win_dependencies.py (working copy)
@@ -1080,7 +1080,7 @@
outfp = subprocess.Popen([os.path.join(jdk_path, 'bin', 'javac.exe'),
'-version'], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT).stdout
- line = outfp.read()
+ line = outfp.read().decode('utf8')
if line:
vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
else:
@@ -1138,7 +1138,7 @@
try:
fp = subprocess.Popen([self.swig_exe, '-version'],
stdout=subprocess.PIPE).stdout
- txt = fp.read()
+ txt = fp.read().decode('utf8')
if txt:
vermatch = re.search(r'^SWIG\ Version\ (\d+)\.(\d+)\.(\d+)', txt, re.M)
else:
@@ -1166,7 +1166,7 @@
try:
fp = subprocess.Popen([self.swig_exe, '-swiglib'],
stdout=subprocess.PIPE).stdout
- lib_dir = fp.readline().strip()
+ lib_dir = fp.readline().decode('utf8').strip()
fp.close()
except OSError:
lib_dir = None
