[issue7559] TestLoader.loadTestsFromName swallows import errors

[Matej Cepl]

Changes by Matej Cepl :


--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray added the comment:

It looks to me like this is complete, so closing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

Its backported in unittest2 0.8.0 which is available on pypi for 2.6+ and 3.2+.

The changes are large enough that I'd hesitate to backport them in cPython 
itself.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Tim Graham]

Changes by Tim Graham timogra...@gmail.com:


--
nosy: +Tim.Graham

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Domen Kožar]

Domen Kožar added the comment:

Could we backport this one to 3.x and 2.7? It's leads to really bad UX.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray added the comment:

I would love that, but I think the fix depends on a feature.  Robert will know 
for sure.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Roundup Robot]

Roundup Robot added the comment:

New changeset 708b2e656c1d by Robert Collins in branch 'default':
Close #7559: ImportError when loading a test now shown as ImportError.
https://hg.python.org/cpython/rev/708b2e656c1d

--
nosy: +python-dev
resolution:  - fixed
stage: patch review - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

I've updated the patch to try and address the niggling clarity issues from the 
review. Please let me know what you think (and if I hear nothing I'll commit it 
as-is since the review was still ok).

--
Added file: http://bugs.python.org/file37045/issue7559.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Berker Peksag]

Changes by Berker Peksag berker.pek...@gmail.com:


--
nosy: +berker.peksag

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

Thanks for the review, updated patch here - I'll let this sit for a day or two 
for more comments then commit it Monday.

--
Added file: http://bugs.python.org/file37002/issue7559.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Changes by Robert Collins robe...@robertcollins.net:


Added file: http://bugs.python.org/file36974/issue7559.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

Patch polished up and updated - ready for review IMO.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray added the comment:

Thanks for tackling this.  It's been bugging me almost daily this past week, 
but as usual when this bug is in my face I had no time to actually work on a 
fix.

I applied this patch to default,  put an invalid import in test_os, and this is 
the result:

rdmurray@pydev:~/python/p35./python -m unittest test.test_os
Traceback (most recent call last):
  File /home/rdmurray/python/p35/Lib/unittest/loader.py, line 140, in 
loadTestsFromName
module = __import__(module_name)
  File /home/rdmurray/python/p35/Lib/test/test_os.py, line 5, in module
import foobar
ImportError: No module named 'foobar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File /home/rdmurray/python/p35/Lib/runpy.py, line 170, in 
_run_module_as_main
__main__, mod_spec)
  File /home/rdmurray/python/p35/Lib/runpy.py, line 85, in _run_code
exec(code, run_globals)
  File /home/rdmurray/python/p35/Lib/unittest/__main__.py, line 18, in 
module
main(module=None)
  File /home/rdmurray/python/p35/Lib/unittest/main.py, line 92, in __init__
self.parseArgs(argv)
  File /home/rdmurray/python/p35/Lib/unittest/main.py, line 139, in parseArgs
self.createTests()
  File /home/rdmurray/python/p35/Lib/unittest/main.py, line 146, in 
createTests
self.module)
  File /home/rdmurray/python/p35/Lib/unittest/loader.py, line 202, in 
loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
  File /home/rdmurray/python/p35/Lib/unittest/loader.py, line 202, in 
listcomp
suites = [self.loadTestsFromName(name, module) for name in names]
  File /home/rdmurray/python/p35/Lib/unittest/loader.py, line 145, in 
loadTestsFromName
next_attribute, self.suiteClass)
ValueError: need more than 1 value to unpack

I get similar errors if I misspell the module name in on the command line.

From my point of view this is still an improvement over the status quo, but I 
don't think it is what you had in mind :).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

You may need to apply the patch from http://bugs.python.org/issue19746 first as 
well - I was testing with both applied.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray added the comment:

OK, with both patches applied the output looks good.  With a bit of luck I'll 
have some time to actually review the patches in a couple of hours.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

This is what I see in my tree:

E
==
ERROR: test_os (unittest.loader.ModuleImportFailure)
--
Traceback (most recent call last):
  File /home/robertc/work/cpython/Lib/unittest/case.py, line 58, in 
testPartExecutor
yield
  File /home/robertc/work/cpython/Lib/unittest/case.py, line 577, in run
testMethod()
  File /home/robertc/work/cpython/Lib/unittest/loader.py, line 36, in 
testFailure
raise exception
ImportError: Failed to import test module: test_os
Traceback (most recent call last):
  File /home/robertc/work/cpython/Lib/unittest/loader.py, line 146, in 
loadTestsFromName
module = __import__(module_name)
  File /home/robertc/work/cpython/Lib/test/test_os.py, line 5, in module
import broken
ImportError: No module named 'broken'

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

Raced with your comment. Great - and thanks!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

I've just put a patch up for the related issue http://bugs.python.org/issue19746

I'll poke at this one briefly now, since I'm across the related code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins added the comment:

Ok, here is an implementation that I believe covers everything Michael wanted. 
I examined the other patches, and can rearrange my implementation to be more 
like them if desired - but at the heart of this this bug really has two 
requested changes:
 - deferred reporting of error per Michaels request
 - report missing attributes on packages as an ImportError (if one occurred)

and thus my implementation focuses on those changes.

--
Added file: http://bugs.python.org/file36578/issue7559.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Mark Lawrence]

Mark Lawrence added the comment:

Note that this issue is referred to from #15358.

--
nosy: +BreamoreBoy
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Mark Lawrence]

Mark Lawrence added the comment:

Note that #8297 referenced in msg102236 is closed see changeset d84a69b7ba72.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[vila]

Changes by vila v.ladeuil+bugs-pyt...@free.fr:


--
nosy: +vila

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Zachary Ware]

Changes by Zachary Ware zachary.w...@gmail.com:


--
nosy: +zach.ware

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Ezio Melotti]

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
nosy: +ezio.melotti

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Michael Foord added the comment:

My preferred fix is to wrap an exception during import as a test that fails 
instead of an AttributeError. This would definitely be a new feature rather 
than a bugfix - so it could only be in 3.4. 

It could be made available to Python 2.7 through the unittest2 backport.

None of the current patches implement my preferred solution yet.

--
versions: +Python 3.4 -Python 2.7, Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Domen Kožar]

Domen Kožar added the comment:

One relevant use case is the following: 
https://github.com/Pylons/venusian/issues/23

Here the module is supposed to raise an ImportError.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Domen Kožar]

Domen Kožar added the comment:

What can I do to put this forward? It's still an issue in py2.7

--
nosy: +iElectric

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Eric Snow]

Changes by Eric Snow ericsnowcurren...@gmail.com:


--
nosy: +eric.snow

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Julian Berman]

Changes by Julian Berman julian+python@grayvines.com:


--
nosy: +Julian

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Because we don't know if the rest of the test code will adhere to this, we 
might want to consider clearing the cache before each test as well.

Alternatively, we could avoid having to call importlib.invalidate_caches() at 
all (and having to think about for which tests it is necessary) if we do each 
test in a different directory and with a different name for the test package.  
We could do the former as follows:

with support.temp_cwd(support.TESTFN):
dir_name = self.id().split('.')[-1]  # test method name
with support.temp_cwd(dir_name) as cwd:
with support.DirsOnSysPath(cwd):
# Create package and run test.

An approach like this might be less prone to issues that are hard to 
troubleshoot.  I verified that it works.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

That would probably be OK, but I don't see why clearing the cache in those same 
methods (that create directories on the path) would be any harder.  (It isn't 
necessary to clear the cache *afterward*, only before, as far as I can see, 
since the case of a directory not existing that the cache thinks exists should 
be handled correctly by importlib).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

That sounds fine.  I just got the sense from above that there was a desire to 
call invalidate_caches() as few times as possible.

And yes, I agree only before is necessary.  I had just taken what you said 
above literally (that [the cache] should be reset by tests that change what's 
on the path), thinking that you wanted to maintain the principle that tests 
should leave things as they were at the beginning.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Ah, yes, I wasn't clear.  Sorry.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Thanks for figuring that out.  And no, it doesn't matter if it is 
importlib.load_module or __import__, since both are provided by importlib now 
and both use the cache.

It's an interesting question where the cache clear should go.  I *think* it 
should go in the test, based on the idea that the cache is part of the 
environment, and therefore should be reset by tests that change what's on the 
path.  I'm not sure how we'd write an environment monitor for that, since not 
all changes to the import cache need to be reset.  I wonder if it would be 
worth putting a reset into DirsOnSysPath.

--
nosy: +brett.cannon

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Brett Cannon]

Brett Cannon br...@python.org added the comment:

Not sure what DirsOnSysPath is, but I have been only calling 
importlib.invalidate_caches() as needed in order to not slow down tests 
needlessly.

And as for detecting an environment change as necessary, that's essentially 
impossible since it's only needed if something changed between imports which 
would require adding a hook to notice that an import happened *and* a directory 
already covered by sys.path_importer_cache (not sys.path since that doesn't 
cover packages) changed w/o calling invalidate_caches().

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

OK, let's just do it in the individual test, then.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Thanks.  It looks like the issue with the latest patch is caused by side 
effects of calling importlib.import_module().

Working from the patch, I got it to the point where inserting the following 
four lines somewhere in the code--

try:
importlib.import_module('foo__doesnotexist')
except:
pass

caused the exception raised by the following line--

module = importlib.import_module('package_foo2.subpackage.no_exist')

to change from this--

  ...
  File frozen importlib._bootstrap, line 1250, in _find_and_load_unlocked
ImportError: No module named 'package_foo2.subpackage.no_exist'

to this--

  ...
  File /Lib/importlib/_bootstrap.py, line 1257, in _find_and_load_unlocked
raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'package_foo2'

It looks like this issue is cropping up in the tests because the test code 
dynamically adds packages to directories that importlib may already have 
examined.

In the reduced test case I was creating to examine the issue, I found that 
inserting a call to importlib.invalidate_caches() at an appropriate location 
resolved the issue.

Should loadTestsFromName() call importlib.invalidate_caches() in the new patch 
implementation, or should the test code be aware of that aspect of 
loadTestsFromName()'s behavior and be adjusted accordingly (e.g. by creating 
the dynamically-added packages in more isolated directories)?  For backwards 
compatibility reasons, how does loadTestsFromName() currently behave in this 
regard (i.e. does importlib.import_module() behave the same as __import__ with 
respect to caching)?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

I updated the tests to Python3, and attempted to replicate the fix using the 
new importlib qualname support.  Even if it had worked, this would not have 
finished the patch, since Michael wants to generate a failing test instead of 
raising the import error.

However, I'm running into weird problems and am shelving this for the moment.  
The issue is that if I run the tests like this:

   ./python -m unittest test.test_unittest

(or via regrtest) they fail with the wrong name in the error message.  If I run 
them like this:

./python -m unittest unittest.test.test_loader.TestLoader.name of test

the right name is in the message.  I suspect the bug is in the tests, but I'm 
not spotting it.  Maybe someone else will see it.

--
Added file: http://bugs.python.org/file25790/unittest_loader_import_error.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Does fix for issue 1559549 (and the fact that importlib is in) allow for a 
cleaner fix for this?  I've been putting off dealing with this issue in the 
expectation that it would.

To answer the question: the fixed API can go in 3.3.  If we fix this in earlier 
versions the API should probably stay the same there, and since the above is 
3.3 only, the existing patch is probably appropriate there.  

If Michael doesn't pass judgement soon I'll take a look, since this causes me 
problems at least once a week.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Michael Foord mich...@voidspace.org.uk added the comment:

My favoured fix is to catch the exception and generate a failing test that 
re-raises the *original exception* (with traceback) when run. That way a single 
failing module doesn't kill a whole test run (although it does mean later 
feedback about misspelt imports). It also means (the main problem being 
reported here) that unittest no longer masks exceptions whilst importing test 
modules.

This would be a new feature / api change - so it would be Python 3.3 only (but 
it would go into unittest2).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

I'm still getting hit with this.  In what versions is it okay for us to fix the 
bad API, as Michael suggested?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
versions: +Python 3.3 -Python 2.6, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Michael Foord mich...@voidspace.org.uk added the comment:

My thinking on this has evolved a bit. Changing an import error into an 
attribute error is just a bad api. We should just fix the bad api.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Michael, if you have no objection to this patch I'm willing to commit it.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Alex Garel]

Alex Garel alex.ga...@gmail.com added the comment:

May I just add that I also ran into this and give my +1 for any fix :-)

--
nosy: +alexgarel

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Changes by Michael Foord mich...@voidspace.org.uk:


--
assignee: cjerdonek - michael.foord

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Daniel Diniz]

Changes by Daniel Diniz aja...@gmail.com:


--
stage: unit test needed - patch review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Changes by Chris Jerdonek chris.jerdo...@gmail.com:


--
assignee: michael.foord - cjerdonek

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Patch update: added unit test to cover replacing __import__, incorporated R. 
David Murray's suggestion to use test_support.test_cwd(), and overall code 
clean-up.

Also uploaded as Patch 3 to--

http://codereview.appspot.com/810044/show

--
Added file: http://bugs.python.org/file16862/_patch-7559-6.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Thanks for your suggestions on the test code.  I will do that.

It seems like the hard-coded approach would be more brittle.  For example, if 
someone wants to replace __import__ with their own, e.g.

old__import__ = __builtins__.__import__

def __my_logging_import(*args, **kwargs):
print Importing %s... % args[0]  # module name
return old__import__(*args, **kwargs)

__builtins__.__import__ = __my_logging_import

Then the stack traces would be different:

  File /Users/chris_g4/dev/Python/trunk/Lib/unittest/loader.py, line 92, in 
loadTestsFromName
module = __import__('.'.join(parts_copy))
  File unittests.py, line 8, in __my_logging_import
return old__import__(*args, **kwargs)
ImportError: No module named sdasfasfasdf

This causes the unit tests not to pass.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

 I think you can simplify them a bit. For example, you can use assertRaises.

Actually, assertRaises doesn't seem to permit checking error text.  That may be 
one reason why try-except-else is being used instead throughout.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

FYI, there seems to be a bug in the code cited above:

http://twistedmatrix.com/trac/browser/trunk/twisted/python/reflect.py#L382

For example, _importAndCheckStack('package.subpackage.module') raises
_NoModuleFound in the following scenario:

package/subpackage/__init__.py:
   import no_exist

when it should instead raise an ImportError from the buggy __init__.py.

I now think there should be at least a few unit tests to cover this case and a 
couple similar permutations.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Four failing unit tests (context code can use clean-up).

--
Added file: http://bugs.python.org/file16819/_patch-7559-4-unittests.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

The new unit tests pass with this patch (minor clean-up still needed).

--
Added file: http://bugs.python.org/file16822/_patch-7559-5.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Thank you very much for those tests.  I think you can simplify them a bit. For 
example, you can use assertRaises.  You also might be able to use the 
test_support.temp_cwd context manager in your context manager, even though you 
don't need the cwd part.

I've attached an alternate, simpler patch to fix this bug, based on a similar 
fix I did in pydoc.  The disadvantage of my patch is that it contains a 
hardcoding of the name of the function doing the import.  I think this is 
acceptable given the much greater simplicity of my patch.  I may be missing 
some subtlety, though, that the twisted folks know about.  Or perhaps people 
will just find the hardcoding itself objectionable.

--
Added file: http://bugs.python.org/file16829/alternate_import_fix.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Rietveld link: http://codereview.appspot.com/810044/show

This patch changes unittest.TestLoader.loadTestsFromName() so that ImportErrors 
will bubble up when importing from a module with a bad import statement.  
Before the method raised an AttributeError.  The unit test code is taken from a 
patch by Salman Haq.  The patch also includes code adapted from 
http://twistedmatrix.com .

(This is my first patch, so any guidance is greatly appreciated.  Thanks.)

--
Added file: http://bugs.python.org/file16796/_patch-7559-3.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

The unit test passes on trunk for me without the fix applied.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

Thanks, David.  Sorry about that.  The test probably requires one additional 
level of nesting so that parts_copy is not False:

+if not parts_copy or not module_not_found:
 raise

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

This patch implements Michael's suggestion (but not the ErrorHolder part):

http://bugs.python.org/issue7559#msg97462

The unit tests all pass with no change.  If this approach looks good to you, I 
can add a unit test to the patch that checks that this bug has been fixed.

Also, Twisted Matrix's web site doesn't seem to be responding too well at the 
moment, but if I recall correctly, their code has a permissive (MIT?) license 
that should allow a small snippet like this to be copied without taking extra 
steps.

--
Added file: http://bugs.python.org/file16759/_patch-7559-2.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

 I think in both cases the error text should state not just what module was 
 being imported but also what module was being imported from

FYI, I filed the following report partly in response to some of the comments I 
made above:

http://bugs.python.org/issue8297

(regarding the AttributeError not displaying the name of the module from which 
the caller is trying to get the attribute)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Chris Jerdonek]

Chris Jerdonek chris.jerdo...@gmail.com added the comment:

I was also hit by this today.

For the sake of clarity, I will restate two of the scenarios that have been 
mentioned in this discussion:

(1) An ImportError raised whilst importing a module (original issue)
(2) A sub-module not existing.

I think the error text should be better in both cases and not just in case (1).

Currently, both (1) and (2) yield an error like the following:

AttributeError: 'module' object has no attribute 'subpackage1'

But also in case (2), the AttributeError reveals less information than the 
exception that was trapped earlier:

ImportError: No module named subpackage1.subpackage2

I think in both cases the error text should state not just what module was 
being imported but also what module was being imported from -- e.g. 
root_package.subpackage1.subpackage2.  In other words, it should also include 
the leading parts of--

'.'.join(parts_copy)  

In my case, I passed a list of modules to unittest, and it wasn't clear which 
one it was failing on by looking at only the trailing segment.  Thanks.

--
nosy: +cjerdonek

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Michael Foord mich...@voidspace.org.uk added the comment:

Wouldn't this be a backwards incompatible change of tested behaviour though?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Michael Foord mich...@voidspace.org.uk added the comment:

I'm unhappy with a straight change in behaviour because it will break code that 
is currently catching AttributeError.

A slightly less invasive change would be to raise an AttributeError if the 
module doesn't exist, otherwise letting the original error propagate.

That means distinguishing between a module not existing and an ImportError 
raised whilst importing the module. Example code that does this by walking the 
stack: 
http://twistedmatrix.com/trac/browser/trunk/twisted/python/reflect.py#L382

In addition we could add a new method that loads a test from name, returning an 
'ErrorHolder' if loading the test fails. (A TestCase that reraises the original 
error when run - test discovery already does this in fact so that a test module 
failing to load doesn't halt discovery.)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Changes by Michael Foord mich...@voidspace.org.uk:


--
assignee:  - michael.foord

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Salman Haq]

Salman Haq salman@gmail.com added the comment:

Hope everyone had a good new year's. I've attached an updated patch which adds 
a new unittest, test_loadTestsFromNames__badimport. 

Both the new unittests can use better documentation, hopefully one of you can 
help me with that.

--
Added file: http://bugs.python.org/file15728/issuee7559_trunk_patch.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Salman Haq]

Salman Haq salman@gmail.com added the comment:

I'm attaching a patch (against trunk) which I think is a step in the
right direction but I could use some feedback. 

This patch modifies 'loadTestsFromName()' so that it saves and re-raises
an ImportError.

Further this patch introduces a new unittest
(test_loadTestsFromName__badimport) and slightly modifies two existing
unittests (test_loadTestsFromName__unknown_attr_name,
test_loadTestsFromNames__unknown_attr_name) in test_unittest.py. Also, I
think a second new unittest is needed (test_loadTestsFromNames__badimport).

--
keywords: +patch
Added file: http://bugs.python.org/file15686/issuee7559_trunk_patch.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Salman Haq]

Salman Haq salman@gmail.com added the comment:

Line 348 in trunk/Lib/test/test_unittest.py has a test case to
specifically test that in the described situation, the test returns an
AttributeError. Should this test be changed so that it passes if the
exception is in fact an ImportError?

   def test_loadTestsFromName__unknown_attr_name(self):
loader = unittest.TestLoader()

try:
loader.loadTestsFromName('unittest.sdasfasfasdf')
except AttributeError, e:
self.assertEqual(str(e), 'module' object has no attribute
'sdasfasfasdf')
else:
self.fail(TestLoader.loadTestsFromName failed to raise
AttributeError)

--
nosy: +slmnhq

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

New submission from Robert Collins robe...@robertcollins.net:

Say I have a test module test_foo, which fails to import with
ImportError. A reason for this might be a misspelt import in that module.

TestLoader().loadTestsFromName swallows the import error and instead
crashes with:
  File /usr/lib/python2.6/unittest.py, line 584, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_foo'

A better thing to do would be to keep the import error and if the next
probe is an Attribute error, reraise the import error.
An alternative would be to return a test which would then reraise the
import error permitting the test suite to be constructed and execute but
still reporting the error.

--
components: Library (Lib)
messages: 96770
nosy: rbcollins
severity: normal
status: open
title: TestLoader.loadTestsFromName swallows import errors

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Could you provide a more complete recipe for reproducing the problem,
please?  I created a test_foo.py containing 'import blert', and running
python -m unittest test_foo does not mask the import error for blert in
loadTestsFromName:

...
  File /usr/lib/python2.6/unittest.py, line 576, in loadTestsFromName
module = __import__('.'.join(parts_copy))
  File test_foo.py, line 1, in module
import blert
ImportError: No module named blert

--
nosy: +michael.foord, r.david.murray
priority:  - normal
stage:  - test needed
type:  - behavior
versions: +Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins robe...@robertcollins.net added the comment:

mkdir thing
touch thing/__init__.py
echo import blert  thing/test_foo.py
python -m unittest thing.test_fooTraceback (most recent call last):
  File /usr/lib/python2.6/runpy.py, line 122, in _run_module_as_main
__main__, fname, loader, pkg_name)
  File /usr/lib/python2.6/runpy.py, line 34, in _run_code
exec code in run_globals
  File /usr/lib/python2.6/unittest.py, line 875, in module
main(module=None)
  File /usr/lib/python2.6/unittest.py, line 816, in __init__
self.parseArgs(argv)
  File /usr/lib/python2.6/unittest.py, line 843, in parseArgs
self.createTests()
  File /usr/lib/python2.6/unittest.py, line 849, in createTests
self.module)
  File /usr/lib/python2.6/unittest.py, line 613, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
  File /usr/lib/python2.6/unittest.py, line 584, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'test_foo'

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Thank you.  I can reproduce this on trunk as well.  I'm leaving stage
set to test needed because we need to turn this into a unit test.

--
keywords: +easy
versions: +Python 2.7, Python 3.1, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Note: this problem is similar in some ways to issue 5230, and a similar
solution might be appropriate (or might not :).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Michael Foord]

Michael Foord mich...@voidspace.org.uk added the comment:

I'll try and look at both these issues in the next few days unless one
of you beats me to it. :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7559] TestLoader.loadTestsFromName swallows import errors

[Robert Collins]

Robert Collins robe...@robertcollins.net added the comment:

I'm scratching an itch at the moment, I just noted this in passing ;)

I'm partial to the 'turn it into a fake test case' approach, its what I
would do if I get to it first.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue7559
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com