Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
Hi,
yes, I'd really appreciate such a modification. Then I could delete
the regarding code in my rod.recipe.appengine, which provides a dummy
implementation of imp.load_dynamic.
Tim, do you have a list of the concerning packages? I could help
making these changes. A good reason to file an application for commit
rights :-)
Regards
Tobias
On 03.07.2009, at 02:39, Tim Hoffman wrote:
> Hi all
>
> Can I make a suggestion that would make a couple of modules more gae
> friendly ;-)
> zope.interface and zope.i18nmessageid have 'c' optimisations which
> obviously don't work under app engine.
>
>
> When these modules are imported you get the following exception.
>
> gae/1.333250465889549129/zope/i18nmessageid/
> _zope_i18nmessageid_message.py",
> line 6, in __bootstrap__
>imp.load_dynamic(__name__,__file__)
> File "/base/python_dist/lib/python2.5/py_imp.py", line 116, in
> load_dynamic
>raise NotImplementedError('This function is not supported on App
> Engine.')
>
> Unfortunately the code (in this case in zope.i18nmessageid.Message)
> does the following which doesn't deal with the Exception
> raised by the app engine runtime.
>
>
> try:
>from _zope_i18nmessageid_message import Message
> except ImportError:
>pass
>
> To make it more "gae friendly" I think we should change these
> occurrences to
>
> try:
>from _zope_i18nmessageid_message import Message
> except ImportError, NotImplementedError:
>pass
>
> Regards
>
> Tim
> ___
> Zope-Dev maillist - Zope-Dev@zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
Tim Hoffman wrote:
> gae/1.333250465889549129/zope/i18nmessageid/_zope_i18nmessageid_message.py",
> line 6, in __bootstrap__
> imp.load_dynamic(__name__,__file__)
> File "/base/python_dist/lib/python2.5/py_imp.py", line 116, in load_dynamic
> raise NotImplementedError('This function is not supported on App Engine.')
This actually feels like a bug in GAE to me... This should be an
ImportError...
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
I suppose that depends on your point of view.
They exception is actually saying that imp.load_dynamic is not
implemented, not that it can't load the dynamic module so I would say
in my opinion that it isn't a bug.
Rgds
Tim
On Fri, Jul 3, 2009 at 4:38 PM, Chris Withers wrote:
> Tim Hoffman wrote:
>> gae/1.333250465889549129/zope/i18nmessageid/_zope_i18nmessageid_message.py",
>> line 6, in __bootstrap__
>> imp.load_dynamic(__name__,__file__)
>> File "/base/python_dist/lib/python2.5/py_imp.py", line 116, in load_dynamic
>> raise NotImplementedError('This function is not supported on App
>> Engine.')
>
> This actually feels like a bug in GAE to me... This should be an
> ImportError...
>
> Chris
>
> --
> Simplistix - Content Management, Zope & Python Consulting
> - http://www.simplistix.co.uk
> ___
> Zope-Dev maillist - zope-...@zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope )
>
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
On 03.07.2009, at 10:38, Chris Withers wrote:
> Tim Hoffman wrote:
>> gae/1.333250465889549129/zope/i18nmessageid/
>> _zope_i18nmessageid_message.py",
>> line 6, in __bootstrap__
>>imp.load_dynamic(__name__,__file__)
>> File "/base/python_dist/lib/python2.5/py_imp.py", line 116, in
>> load_dynamic
>>raise NotImplementedError('This function is not supported on App
>> Engine.')
>
> This actually feels like a bug in GAE to me... This should be an
> ImportError...
Sorry, I don't agree. We assume an imp.load_dynamic function from what
we read in the python documentation. But the implementation in GAE
does not provide the expected api. So the NotImplementedError seems
absolutely correct to me (see
http://www.python.org/doc/2.5.2/lib/module-exceptions.html#l2h-114)
.
Tobias
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
Tobias Rodäbel wrote:
> Sorry, I don't agree. We assume an imp.load_dynamic function from what
> we read in the python documentation. But the implementation in GAE
> does not provide the expected api. So the NotImplementedError seems
> absolutely correct to me (see
> http://www.python.org/doc/2.5.2/lib/module-exceptions.html#l2h-114)
Ahhh, I see, sorry, misread where the exception was coming from.
Surely the try:except for the NotImplementedError should be rwapped just
around the call to imp.load_dynamic, then?
ie:
try:
imp.load_dynamic(__name__,__file__)
except NotImplementedError:
raise ImportError('load_dynamic not available so C extension cannot
be used')
cheers,
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
On Jul 2, 2009, at 8:39 PM, Tim Hoffman wrote:
> Hi all
>
> Can I make a suggestion that would make a couple of modules more gae
> friendly ;-)
> zope.interface and zope.i18nmessageid have 'c' optimisations which
> obviously don't work under app engine.
>
>
> When these modules are imported you get the following exception.
>
> gae/1.333250465889549129/zope/i18nmessageid/
> _zope_i18nmessageid_message.py",
> line 6, in __bootstrap__
>imp.load_dynamic(__name__,__file__)
> File "/base/python_dist/lib/python2.5/py_imp.py", line 116, in
> load_dynamic
>raise NotImplementedError('This function is not supported on App
> Engine.')
This bootstrapping code is generated by setuptools to deal with
loading extension modules in zip files. It does some unholy things to
make this work. I'd really like to find a way to suppress this.
In general, I find deploying as zip files to be an anti-feature. In
seems especially insane when extension modules are involved.
> Unfortunately the code (in this case in zope.i18nmessageid.Message)
> does the following which doesn't deal with the Exception
> raised by the app engine runtime.
You mean with the exception raised by the setuptools wrapper code.
> try:
>from _zope_i18nmessageid_message import Message
> except ImportError:
>pass
>
> To make it more "gae friendly" I think we should change these
> occurrences to
>
> try:
>from _zope_i18nmessageid_message import Message
> except ImportError, NotImplementedError:
>pass
This might be an ok work around.
How hard would it be to just remove the wrapper script when deploying
to app engine?
Jim
--
Jim Fulton
Zope Corporation
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
Hi Jim That is what I am doing now, the problem I see though is when someone upates (via buildout etc a) module they need to remember to remove the recreated file before redeploying the app, unfortunately I am not sure I see a way reliable way of specifying that these files should be deleted in any automatic sense. At least an ignore in the app.yaml isn't feasible. Tim ... > > > This might be an ok work around. > > How hard would it be to just remove the wrapper script when deploying > to app engine? > > Jim > > -- > Jim Fulton > Zope Corporation > > > ___ > Zope-Dev maillist - zope-...@zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
On Jul 3, 2009, at 6:25 AM, Tim Hoffman wrote: > Hi Jim > > That is what I am doing now, the problem I see though is when someone > upates (via buildout etc a) module they need to remember to > remove the recreated file before redeploying the app, unfortunately I > am not sure I see a way reliable way of specifying that these files > should be deleted in any automatic sense. At least an ignore in the > app.yaml isn't feasible. I'm sure there's a way to do it. It will just take some digging. Maybe there's a way to tell setuptools not to do it or maybe there's enough meta data laying around for buildout to undo it. For example, the EGG-INFO/SOURCES.txt has a list of the original files in the distribution, so it should be easy enough to figure out where these extra bogus files are. Jim -- Jim Fulton Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
On 03.07.2009, at 12:33, Jim Fulton wrote: > On Jul 3, 2009, at 6:25 AM, Tim Hoffman wrote: > >> Hi Jim >> >> That is what I am doing now, the problem I see though is when someone >> upates (via buildout etc a) module they need to remember to >> remove the recreated file before redeploying the app, >> unfortunately I >> am not sure I see a way reliable way of specifying that these files >> should be deleted in any automatic sense. At least an ignore in the >> app.yaml isn't feasible. > > > I'm sure there's a way to do it. It will just take some digging. > Maybe there's a way to tell setuptools not to do it or maybe there's > enough meta data laying around for buildout to undo it. For example, > the EGG-INFO/SOURCES.txt has a list of the original files in the > distribution, so it should be easy enough to figure out where these > extra bogus files are. Great idea to examine EGG-INFO/SOURCES.txt, Jim! I just added this to my recipe (http://pypi.python.org/pypi/rod.recipe.appengine) and got rid of the horrible monkey patch for imp.load_dynamic. Now it does exactly what Tim wants and excludes the optional c extension stuff (plus .pyo, and .pyc files) in a gae buildout. I'm planning to add an option for not deploying the additional packes as a zip archive if the gae file limit will not be exceeded. Cheers, Tobias ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
Hi Tobias Cool Just something to note google recently upped the file limit to 3000 so don't be too agressive generating those zips ;-) T > > Great idea to examine EGG-INFO/SOURCES.txt, Jim! I just added this to > my recipe (http://pypi.python.org/pypi/rod.recipe.appengine) and got > rid of the horrible monkey patch for imp.load_dynamic. Now it does > exactly what Tim wants and excludes the optional c extension stuff > (plus .pyo, and .pyc files) in a gae buildout. I'm planning to add an > option for not deploying the additional packes as a zip archive if the > gae file limit will not be exceeded. > > Cheers, > Tobias > > ___ > Zope-Dev maillist - zope-...@zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Jim Fulton wrote:
> On Jul 2, 2009, at 8:39 PM, Tim Hoffman wrote:
>
>> Hi all
>>
>> Can I make a suggestion that would make a couple of modules more gae
>> friendly ;-)
>> zope.interface and zope.i18nmessageid have 'c' optimisations which
>> obviously don't work under app engine.
>>
>>
>> When these modules are imported you get the following exception.
>>
>> gae/1.333250465889549129/zope/i18nmessageid/
>> _zope_i18nmessageid_message.py",
>> line 6, in __bootstrap__
>>imp.load_dynamic(__name__,__file__)
>> File "/base/python_dist/lib/python2.5/py_imp.py", line 116, in
>> load_dynamic
>>raise NotImplementedError('This function is not supported on App
>> Engine.')
I think the suggestion to remap the NotImplementedError to an Import
error is sensible (in that __bootstrap__ method).
> This bootstrapping code is generated by setuptools to deal with
> loading extension modules in zip files. It does some unholy things to
> make this work. I'd really like to find a way to suppress this.
>
> In general, I find deploying as zip files to be an anti-feature. In
> seems especially insane when extension modules are involved.
The only way to deploy *any* non-toy app on GAE is via zip file(s) (I
would think a single big one would be best), due to their limit on the
number of files.
Tres.
- --
===
Tres Seaver +1 540-429-0999 tsea...@palladion.com
Palladion Software "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFKTkAH+gerLs4ltQ4RAhazAJ4itN+wO1rPwXPyLIEIaNF6VKSScgCdGyfd
OxfF8AUnMYT09/L2B5RF9O0=
=RGfd
-END PGP SIGNATURE-
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] A suggestion for changing exception handler when loading 'c' optimisations to make modules more friendly for GAE
On 7/3/09 1:29 PM, Tres Seaver wrote: >> In general, I find deploying as zip files to be an anti-feature. In >> seems especially insane when extension modules are involved. > > The only way to deploy *any* non-toy app on GAE is via zip file(s) (I > would think a single big one would be best), due to their limit on the > number of files. Actually they've raised the limit to 3000 files now, which is more realistic (previously it was impossible to deploy even minimal apps without zipping library eggs, now the zipping is largely unnecessary if you keep your dependencies sane). - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
