[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[Benjamin Peterson]

Benjamin Peterson  added the comment:

Applied in r70587.

--
resolution:  -> accepted
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

STINNER Victor  added the comment:

Oh, I see that setup.py uses warnings and so linecache is loaded by 
setup.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

STINNER Victor  added the comment:

@benjamin.peterson: The second version of my patch works correctly 
with the bootstraping.

> I also think we should consider hard adding more modules 
> that are loaded at startup time to py3k already huge list.

linecache is not loaded at startup time in py3k! I see that linecache 
is loaded by the warnings module, but the warnings module 
(Lib/warnings.py) is not loaded at startup. It was maybe the case with 
Python 2.x or older version of Python 3.x?

With my patch, loading linecache loads 2 extra modules: tokenize and 
token. It only impacts code using directly linecache or the warnings 
module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

STINNER Victor  added the comment:

> I also think we should consider hard adding more modules 
> that are loaded at startup time to py3k already huge list.

My patch uses tokenize modules in setup.py bootstrap. But it doesn't 
affect Python classic usage "python" or "python myscript.py".

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
priority:  -> normal
stage:  -> patch review
versions: +Python 3.1 -Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file11678/linecache_refactor.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

STINNER Victor  added the comment:

It was easy to remove the itertools dependency: only itertools.chain() 
was really used and a simple "for ...: yield" is enough to get the 
same behaviour (test_tokenize.py runs fine). With the new version of 
the patch, the bootstrap works correctly.

Added file: http://bugs.python.org/file12333/linecache_refactor-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[Benjamin Peterson]

Benjamin Peterson  added the comment:

I had to revert this because tokenize imports itertools. This is a
problem when building (setup.py) because itertools doesn't exist, yet. I
also think we should consider hard adding more modules that are loaded
at startup time to py3k already huge list.

--
resolution: accepted -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[Benjamin Peterson]

Benjamin Peterson  added the comment:

Applied in r67713.

--
nosy: +benjamin.peterson
resolution:  -> accepted
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

STINNER Victor  added the comment:

"This patch looks good." ok and then?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc <[EMAIL PROTECTED]> added the comment:

This patch looks good.

--
nosy: +amaury.forgeotdarc

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

Changes by STINNER Victor <[EMAIL PROTECTED]>:


Removed file: http://bugs.python.org/file11679/tokenize_bom_utf8.patch

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

STINNER Victor <[EMAIL PROTECTED]> added the comment:

I wrote a different (and better) patch for tokenize module: moved to 
the issue4021.

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

Changes by STINNER Victor <[EMAIL PROTECTED]>:


Added file: http://bugs.python.org/file11679/tokenize_bom_utf8.patch

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4016] improve linecache: reuse tokenize.detect_encoding() and io.open()

[STINNER Victor]

New submission from STINNER Victor <[EMAIL PROTECTED]>:

linecache uses it own code to detect a Python script encoding whereas 
a function tokenize.detect_encoding() already exists. It does also 
convert bytes => unicode conversion of the file lines whereas open() 
already supports this feature.

--
components: Library (Lib)
files: linecache_refactor.patch
keywords: patch
messages: 74165
nosy: haypo
severity: normal
status: open
title: improve linecache: reuse tokenize.detect_encoding() and io.open()
versions: Python 3.0
Added file: http://bugs.python.org/file11678/linecache_refactor.patch

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com