[issue24960] Can't use lib2to3 with embeddable zip file.

[Lukasz]

Lukasz  added the comment:

I have simmilar problem but im using python 2.7 on android. So I think I cant 
use this solution 
My problem:
https://bugs.python.org/issue33208

--
nosy: +Lukasz00500

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 85f71aa9d6f834c7d64e979009c8fda0f19b585d by Benjamin Peterson 
(Miss Islington (bot)) in branch '3.6':
make PatternCompiler use the packaged grammar if possible (more bpo-24960) 
(GH-5034) (#5036)
https://github.com/python/cpython/commit/85f71aa9d6f834c7d64e979009c8fda0f19b585d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 417f76a20e93044e422fb328a2d8e200fc6df295 by Benjamin Peterson 
(Miss Islington (bot)) in branch '2.7':
make PatternCompiler use the packaged grammar if possible (more bpo-24960) 
(GH-5034) (#5037)
https://github.com/python/cpython/commit/417f76a20e93044e422fb328a2d8e200fc6df295


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4922

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4921

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset e5f7dccefaa8d97ab53b3051acbb4a4d49379dc4 by Benjamin Peterson in 
branch 'master':
make PatternCompiler use the packaged grammar if possible (more bpo-24960) 
(#5034)
https://github.com/python/cpython/commit/e5f7dccefaa8d97ab53b3051acbb4a4d49379dc4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Change by Benjamin Peterson :


--
pull_requests: +4920

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Change by Benjamin Peterson :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 770a802df7057bcf0d72bbc01026fac8d396e1b3 by Benjamin Peterson in 
branch '2.7':
[2.7] bpo-24960: use pkgutil.get_data in lib2to3 to read pickled grammar files 
(GH-4977) (#4980)
https://github.com/python/cpython/commit/770a802df7057bcf0d72bbc01026fac8d396e1b3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset c1b8eb8006ed79bbf91dc7d89918f39cc10b4fe6 by Benjamin Peterson 
(Miss Islington (bot)) in branch '3.6':
bpo-24960: use pkgutil.get_data in lib2to3 to read pickled grammar files 
(GH-4977) (#4979)
https://github.com/python/cpython/commit/c1b8eb8006ed79bbf91dc7d89918f39cc10b4fe6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Change by Benjamin Peterson :


--
pull_requests: +4869

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4868

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 8a5877165e993afb2633cd48da5222326d3f6e0e by Benjamin Peterson in 
branch 'master':
bpo-24960: use pkgutil.get_data in lib2to3 to read pickled grammar files (#4977)
https://github.com/python/cpython/commit/8a5877165e993afb2633cd48da5222326d3f6e0e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Benjamin Peterson]

Change by Benjamin Peterson :


--
keywords: +patch
pull_requests: +4867
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Gregory P. Smith]

Gregory P. Smith added the comment:

re: Paul's "I'm not sure I see why you'd want to run lib2to3 with the embedded 
distribution anyway. Surely you'd do any 2to3 conversion in a normal 
development installation"

lib2to3 is a library useful for doing things with Python code.  Not just a 
stand alone tool.

Kirk: Could you please post your change as a pull request on 
https://github.com/python/cpython/ rather than attaching individual files here. 
 a full file isn't a reviewable change.

Also, Grammar.txt is loaded and saved as a cached pickle file for speed.  When 
packaging up an embedded stdlib bundle I recommend shipping that pickle.  
(loaded in pgen2/grammar.py).

both probably need to use 
https://docs.python.org/3.6/library/pkgutil.html#pkgutil.get_data to read the 
file out of the package rather than assuming it lives on a filesystem.

using the zipfile module within lib2to3 to try and figure out when it should 
look somewhere other than a filesystem is the wrong approach.

--
nosy: +gregory.p.smith
stage:  -> needs patch
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Kirk Sayre]

Kirk Sayre added the comment:

Here is a more robust version of my local fix for this problem.

--
Added file: http://bugs.python.org/file46869/pgen.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Kirk Sayre]

Kirk Sayre added the comment:

I've been fighting with this issue today trying to import networkx in the 
embedded Python shipped with IDA-Pro. I have implemented a local fix to pgen.py 
that allows networkx to import and be used in IDAPython scripts. I added a 
check to see if a .zip archive appears in the path of the file loaded in the 
constructor and if so I use the zipfile package to open the archived file 
rather than the standard file open() function.

I've uploaded the modified pgen.py in case any one is interested.

--
nosy: +kirk.d.sayre
versions: +Python 2.7 -Python 3.5
Added file: http://bugs.python.org/file46866/pgen.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Philippe Pinard]

Philippe Pinard added the comment:

As Sébastien Taylor, I ran into the same problem. The workaround I found was to 
unzip the content of python35.zip and put it in the Lib/ folder.

--
nosy: +ppinard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Sébastien Taylor]

Sébastien Taylor added the comment:

Are there existing tools for bundling the python35.zip (and adding additional 
packages)?  I personally ran into this issue by using pip to add additional 
modules to my embeddable python but if there's a better way to migrate from 
devel to embeddable python I'd rather not have to use pip for this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Paul Moore]

Paul Moore added the comment:

I'm not sure I see why you'd want to run lib2to3 with the embedded distribution 
anyway. Surely you'd do any 2to3 conversion in a normal development 
installation, and then simply bundle the resulting converted scripts with your 
application and the embedded interpreter?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Sébastien Taylor]

Changes by Sébastien Taylor :


--
nosy: +Sébastien Taylor

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24960] Can't use lib2to3 with embeddable zip file.

[Nick Coghlan]

Nick Coghlan added the comment:

(issue title updated to indicate this appears to be lib2to3 related and added 
Benjamin Peterson to the nosy list)

The build process already generates a _sysconfigdata.py file in order to pass 
along build time details to the sysconfig module in a way that isn't dependent 
on having otherwise build-time only files accessible at runtime.

Doing something similar with a _lib2to3grammar.py file seems like it could be a 
reasonable approach to making lib2to3 zip archive friendly, but I'm not 
familiar enough with pgen2's internals to say how feasible that would be.

--
nosy: +benjamin.peterson
title: Can't use pip or easy_install with embeddable zip file. -> Can't use 
lib2to3 with embeddable zip file.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com