[issue24263] unittest cannot load module whose name starts with Unicode

[Toshio Kuratomi]

Toshio Kuratomi  added the comment:

>From the description, I think the bug is that filenames that *begin* with 
>non-ascii are not searched for tests.  Looking at the test_dir.tar.gz 
>contents,  this is the test case that I'd use:

Broken:

$ python3 -m unittest discover -vv -p '*.py'
test_走 (tests試驗.Test試驗.試驗) ... ok
test_走 (tests試驗.test試驗.試驗) ... ok

--
Ran 2 tests in 0.000s

OK

Corrected:
$ /srv/python/cpython/python -m unittest discover -vv -p '*.py'
test_走 (tests試驗.Test試驗.試驗) ... ok
test_走 (tests試驗.test試驗.試驗) ... ok
test_走 (tests試驗.試驗.試驗) ... ok

--
Ran 3 tests in 0.000s

OK


isidentifier() is used because filenames to be discovered must be importable 
and thus valid identifiers:  
https://docs.python.org/3/library/unittest.html#test-discovery

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[STINNER Victor]

STINNER Victor  added the comment:

What is the current error on test_dir.tar.gz? I'm not sure which problem is 
trying to be solved here.

Why does PR 13149 use str.isidentifier() method? unittest doesn't allow 
arbitrary Unicode in filenames?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[anthony shaw]

anthony shaw  added the comment:

thanks, will wait for a review from Serhiy, Rbcollins or ezio

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Toshio Kuratomi]

Toshio Kuratomi  added the comment:

I've opened a new PR at https://github.com/python/cpython/pull/13149 with the 
commit from https://github.com/python/cpython/pull/1338 and some additional 
changes to address the review comments given by serhiy.storchaka and rbcollins

--
nosy: +a.badger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Toshio Kuratomi]

Change by Toshio Kuratomi :


--
pull_requests: +13064
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[anthony shaw]

anthony shaw  added the comment:

The original PR refers to a branch that no longer exists, but the behaviour 
documented still applies to master. There were some changes to the test loader, 
but none that fixed this issue.

--
nosy: +anthonypjshaw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Louie Lu]

Louie Lu added the comment:

Add PR: https://github.com/python/cpython/pull/1338/


rbcollins: Need for help to review the patch, I think that both `$thing` and 
`$thing.py` can't be used in python (and for UNIX dir), and `\u2603` (☃) though 
can do something like `☃.py`, but it is not a valid identifier in python, too.

--
nosy: +louielu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Louie Lu]

Changes by Louie Lu :


--
pull_requests: +1449

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Robert Collins]

Robert Collins added the comment:

sih4sing5hong5  - I think we do need a test in fact - it can be done using 
mocks, but right now I think the patch has a bug - it looks for isidentifier on 
$thing.py, but not on just $thing (which we need to do to handle packages, vs 
modules).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Ezio Melotti]

Changes by Ezio Melotti :


--
components: +Library (Lib)
keywords: +easy
stage: patch review -> test needed
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Robert Collins]

Robert Collins added the comment:

I'm torn on whether this needs a test or not. It would be hard to regress, but 
testing this properly really wants hypothesis with a 
valid-python-identifier-strategy.

I think on balance we do need one.

So - we need a test in test_discover that mocks the presence of a file with a 
name containing e.g. \u2603.

--
stage:  - patch review

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Robert Collins]

Robert Collins added the comment:

Thank you very much for writing your patch in backwards compatible style - it 
will make backporting to unittest2 much easier.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[sih4sing5hong5]

sih4sing5hong5 added the comment:

Thank you.
I updated my patch in `VALID_MODULE_NAME.patch`.

--
Added file: http://bugs.python.org/file39789/VALID_MODULE_NAME.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[sih4sing5hong5]

Changes by sih4sing5hong5 ihc...@gmail.com:


Removed file: http://bugs.python.org/file39778/VALID_MODULE_NAME.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

r'[^\W\d]\w*' doesn't match all valid Python identifiers. It would be more 
correct to write the check as:

root, ext = os.path.splitext(basename)
if not (ext == '.py' and root.isidentifier()):
# valid Python identifiers only
return None, False

--
nosy: +serhiy.storchaka
versions: +Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[R. David Murray]

R. David Murray added the comment:

Yes, I bet that regex is left over from python2, where we didn't have 
isidentifier.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[sih4sing5hong5]

sih4sing5hong5 added the comment:

update by adding `except AttributeError:`

--
Added file: http://bugs.python.org/file39794/VALID_MODULE_NAME2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[sih4sing5hong5]

New submission from sih4sing5hong5:

Because VALID_MODULE_NAME is r'[_a-z]\w*\.py$' in unittest/loader.py.

Using r'[^\W\d]\w*\.py$' insteaded.

--
keywords: +patch
title: Why VALID_MODULE_NAME in unittest/loader.py is r'[_a-z]\w*\.py$' not 
r'\w+\.py$' ? - unittest cannot load module whose name starts with Unicode
Added file: http://bugs.python.org/file39778/VALID_MODULE_NAME.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[Robert Collins]

Robert Collins added the comment:

Are the module names valid in import statements?

it would help if you could perhaps attach a little tar/zip file with an example 
failure.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[sih4sing5hong5]

sih4sing5hong5 added the comment:

There is an attached file for examples.

I ran
{{{
cd test_dir
python -m unittest -v
}}}

and got
Ran 1 test in 0.000s

--
Added file: http://bugs.python.org/file39779/test_dir.tar.gz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24263] unittest cannot load module whose name starts with Unicode

[sih4sing5hong5]

sih4sing5hong5 added the comment:

By the way, I ran with Python 3.4.0.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24263
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com