On Sat, Jun 16, 2012 at 11:59 -0400, Ian Miers wrote:
> So these issues only happen on OSX and as I said,they come and go and
> change,which is why I am skeptical it's our code. Actually if I had to
> guess I'd say its something with python3 on OSX Lion, but so far we've only
> seen the issues in test runs, not when attempting to manually recreate the
> issue.
>
> Regarding import os.path error, I got it again. Trying to import the module
> containing that import after that error caused a bus error. However all
> subsequent imports worked fine. The strange thing is that the pyc file
> didn't change ( or at least its md5 sum didn't) and the modification date
> for all of the pyc files for that package remain unchanged.
>
>
> python3.2 -m pytest --assert=plain seemed to work for a little bit, but
> then we started getting the same intermittent changing errors.
With --assert=plain pytest should not interfere with anything related
to importing. Did you get the strange error with "import os.path"
in this configuration? Are you using any plugins (see output of
py.test --version)?
The below error may in principle be a doctest/pytest bug with
python3.2 although it would be strange if it only happened sometimes.
Holger
> Allmost all runs with assert=reinterp , including after removing all
> __pychache__ and recompiling the c extensions produce the INTERNALERROR
> below.
>
>
>
> Thanks again,
> Ian
>
> python3.2 -m pytest --assert=reinterp
> ============================= test session starts
> ==============================
> platform darwin -- Python 3.2.3 -- pytest-2.2.4
> collected 232 items
>
> schemes/__init__.py .
> schemes/chamhash_adm05.py .
> schemes/chamhash_rsa_hw09.py .
> schemes/encap_bchk05.py .
> schemes/pk_fre_ccv11.py .
> schemes/pk_vrf.py .
> schemes/protocol_cns07.py .
> schemes/protocol_schnorr91.py .
> schemes/sigma1.py .
> schemes/sigma2.py .
> schemes/sigma3.py .
> schemes/abenc/__init__.py .
> schemes/abenc/abenc_adapt_hybrid.py F
> schemes/abenc/abenc_bsw07.py .
> schemes/abenc/abenc_lsw08.py .
> schemes/abenc/abenc_waters09.py .
> schemes/abenc/kpabenc_adapt_hybrid.py F
> schemes/commit/__init__.py .
> schemes/commit/commit_gs08.py .
> schemes/commit/commit_pedersen92.py .
> schemes/dabenc/__init__.py .
> schemes/dabenc/dabe_aw11.py
> INTERNALERROR> Traceback (most recent call last):
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> line 74, in wrap_session
> INTERNALERROR> doit(config, session)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> line 106, in _main
> INTERNALERROR> config.hook.pytest_runtestloop(session=session)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 421, in __call__
> INTERNALERROR> return self._docall(methods, kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 432, in _docall
> INTERNALERROR> res = mc.execute()
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 350, in execute
> INTERNALERROR> res = method(**kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> line 119, in pytest_runtestloop
> INTERNALERROR> item.config.hook.pytest_runtest_protocol(item=item,
> nextitem=nextitem)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 421, in __call__
> INTERNALERROR> return self._docall(methods, kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 432, in _docall
> INTERNALERROR> res = mc.execute()
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 350, in execute
> INTERNALERROR> res = method(**kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> line 61, in pytest_runtest_protocol
> INTERNALERROR> runtestprotocol(item, nextitem=nextitem)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> line 68, in runtestprotocol
> INTERNALERROR> reports.append(call_and_report(item, "call", log))
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> line 99, in call_and_report
> INTERNALERROR> report = hook.pytest_runtest_makereport(item=item,
> call=call)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/main.py",
> line 141, in call_matching_hooks
> INTERNALERROR> return hookmethod.pcall(plugins, **kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 425, in pcall
> INTERNALERROR> return self._docall(methods, kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 432, in _docall
> INTERNALERROR> res = mc.execute()
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 350, in execute
> INTERNALERROR> res = method(**kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/capture.py",
> line 173, in pytest_runtest_makereport
> INTERNALERROR> rep = __multicall__.execute()
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/core.py",
> line 350, in execute
> INTERNALERROR> res = method(**kwargs)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/runner.py",
> line 187, in pytest_runtest_makereport
> INTERNALERROR> longrepr = item.repr_failure(excinfo)
> INTERNALERROR> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pytest-2.2.4-py3.2.egg/_pytest/doctest.py",
> line 44, in repr_failure
> INTERNALERROR> lineno = test.lineno + example.lineno + 1
> INTERNALERROR> TypeError: unsupported operand type(s) for +: 'NoneType' and
> 'int'
>
> ===================== 2 failed, 19 passed in 3.42 seconds
> ======================
>
>
>
>
> On Sat, Jun 16, 2012 at 4:19 AM, holger krekel <hol...@merlinux.eu> wrote:
>
> > Hi Ian,
> >
> > On Fri, Jun 15, 2012 at 20:32 -0400, Ian Miers wrote:
> > > Hi, I just started using pytest. It's lovely.
> > > The TLDR on this is we are getting intermittent non reproducible errors
> > > that change and sometimes disappear between test runs like the
> > following :
> > > > import os.path
> > > E TypeError: __import__() argument 1 must be str without null bytes,
> > not
> > > str
> >
> > Looks odd. pytest does not override __import__ but it does by default
> > use a PEP302 compliant module loader. If you use
> >
> > py.test --assert=reinterpret # or --assert=plain
> >
> > do the errors go away? If so that points to a problem in pytest's module
> > loader or some interaction problem with python3.2.
> >
> > If changing the assertion mode still leads to errors then i strongly
> > suspect it's other parts of the code you are running. I'd then suggest
> > to check if something in your environment modifies __import__
> > e.g. by writing a test that checks/prints out __import__?
> >
> > best,
> > holger
> >
> > > Longer version:
> > > We've been getting buggy results out of test runs on OSX Lion with python
> > > 3.2.3 and py.test version 2.2.4. Specifically we've been getting what
> > > appear to be false-positive test failures that change from run to run
> > and
> > > cannot be reproduced by running some code ourselves or in the case of
> > > doctest manually running python3.2 -m doctest file.
> > >
> > > Some of these errors will stay around for multiple test runs even after
> > > make clean, etc. Some will change from run to run. All of them
> > > eventually disappeared temporarily and we got a clean test pass, though
> > > how I don't know. Morever, the problems cropped up again. All of this was
> > > with no code changes and code known to work on ubuntu and partially
> > > manually tested on OSX.
> > >
> > > Ordinarly I'd say there was something wrong with our code. However, some
> > of
> > > the errors are vanishingly unlikely. Claims that modules don't exist when
> > > they do and are importable via python3.2 -c "from foo.bar.baz import
> > narf"
> > > and such.
> > >
> > > The most glaringly, however, is this gem:
> > > charm/toolbox/pairinggroup.py:3: in <module>
> > > > import os.path
> > > E TypeError: __import__() argument 1 must be str without null bytes,
> > not
> > > str
> > >
> > > We also got a lovely bug where it appeared __pycache__ was corrupted
> > > during test runs. On an initial run, we could import a function from a
> > > python c extension. On subsequent runs, it didn't exist. The function was
> > > still in the .so file, as shown by nm, however help(module) returned
> > > function_name#$@^%#$% Function description. Deleting __pychache__ folders
> > > resolved it for the next test run but then it came back. It too
> > > disappeared after a couple of test runs never to be seen since.
> > >
> > > Has anyone seen anyhting like this? Are their known issues on OSX with
> > > python ? With pytest? Does anyone have any idea how I might get a better
> > > idea whats going on?
> > >
> > > As an addendum, the latest error I am getting is now :
> > > > test = [hashFn(struct.pack(">%dsI" % (len(seed)), seed, i)) for i in
> > > ran]
> > > E UnicodeEncodeError: 'ascii' codec can't encode character '\x9e' in
> > > position 0: ordinal not in range(128
> > > This actually might be in our code,though again it works on ubuntu and at
> > > one point on OSX. Given that its a pattern that points to python or
> > pytest
> > > doing something to binaries, I'm including it anyway.
> > >
> > > The project is charm, you can see the code on this branch here via
> > > https://github.com/JHUISI/charm/tree/dev. Python was installed via fenc
> > ( I
> > > think, its not my box). The errors happened both with python3.2 -m pytest
> > > and with python3.2 setup.py test. Though it appears more so with the
> > > later.
> > >
> > > Thanks,
> > >
> > > Ian
> >
> > > _______________________________________________
> > > py-dev mailing list
> > > py-dev@codespeak.net
> > > http://codespeak.net/mailman/listinfo/py-dev
> >
> >
_______________________________________________
py-dev mailing list
py-dev@codespeak.net
http://codespeak.net/mailman/listinfo/py-dev
