Re: py2exe + svn - the final drama
In article [EMAIL PROTECTED], David Bolen [EMAIL PROTECTED] wrote: Just [EMAIL PROTECTED] writes: the zipimport module has an attr called _zip_directory_cache, which is a dict you can .clear(). Still, reloading modules is hairy at best, its probably easiest to relaunch your app when the .zip file has changed. Except that he's getting an error during the process exit of the current execution, which is needed to restart. And if he updates to a different copy, there's the bootstrap problem of how to get it back into the standard location for the next restart since his application will need to have it to restart in the first place. Right. It sounds like a module is imported at exit that wasn't imported before. If that's the case, it may help to make sure all modules needed for exit are imported beforehand. Just -- http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
Timothy Smith wrote:
ok, i am updating my program from my svn - this works, however, i am
getting the following error when i close the program. the zip file
successfully updates, so i'm curious as to the meaning of this.
Traceback (most recent call last):
File Main.pyo, line 820, in ValidateLogin
File Main.pyo, line 863, in ShowMainFrameItems
File glob.pyo, line 22, in glob
File glob.pyo, line 50, in glob1
File fnmatch.pyo, line 42, in filter
zipimport.ZipImportError: bad local file header in Z:\temp\library.zip
not that once i have finished client.update(''), it has successfully
updated the zipfile, i open a dialoge box saying click ok and restart
program AFTER i click on the above error pops up and my app shuts down
as intended.
ideas?
ok i have done some digging and i cound this
/* Check to make sure the local file header is correct */
fseek(fp, file_offset, 0);
l = PyMarshal_ReadLongFromFile(fp);
if (l != 0x04034B50) {
/* Bad: Local File Header */
PyErr_Format(ZipImportError,
bad local file header in %s,
archive);
fclose(fp);
can anyone explain to me about zip file headers and why it would be
different/incorrect and give me this error?
return NULL;
--
http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
Timothy Smith [EMAIL PROTECTED] writes:
Timothy Smith wrote:
(...)
zipimport.ZipImportError: bad local file header in Z:\temp\library.zip
not that once i have finished client.update(''), it has successfully
updated the zipfile, i open a dialoge box saying click ok and
restart program AFTER i click on the above error pops up and my app
shuts down as intended.
ideas?
ok i have done some digging and i cound this
/* Check to make sure the local file header is correct */
fseek(fp, file_offset, 0);
l = PyMarshal_ReadLongFromFile(fp);
if (l != 0x04034B50) {
/* Bad: Local File Header */
PyErr_Format(ZipImportError,
bad local file header in %s,
archive);
fclose(fp);
can anyone explain to me about zip file headers and why it would be
different/incorrect and give me this error?
Are you perhaps trying to update the zip file in-place while it is still
being used by the application? I'm not sure that's a safe operation. A
quick peek at the same module where I think you found the above code shows
that when a zip importer instance is associated with a zip file, the
directory for that zip file is read in and cached. So the importer is
holding onto offset information for each file based on the contents of the
zip directory at initialization time.
If you then change the file contents (such as updating it with svn), those
offsets will no longer be valid. I then expect that during your process
exit, some bit of code is performing an extra import, which accesses the
wrong (based on the new file contents) portion of the zip file, and the
above safety check prevents it from loading an erroneous set of bytes
thinking its a valid module.
I expect you need to work on a mechanism to update the file
independently of the running copy, and then arrange to have it moved
into place for a subsequent execution. Or find some way to have the
zip importer refresh its directory information or make a new importer
instance once the zip file is updated.
One (untested) thought ... before the update, make a copy of your
current library.zip as some other name, and adjust your sys.path to
reference that name (rather than the default pointer to the main
library.zip that py2exe initializes things with). That should force
any future imports to access the old copy of the zip file and not the
one that svn will be updating. Since you need to leave that zip file
copy in place through the exit (to satisfy any trailing imports),
arrange for your application to check for that copy on startup and
remove it if present.
Or, after looking through import.c handling for zip file imports,
there might be a simpler way. ZIP imports are handled by a
zipimporter installed in sys.path_hooks, and once a specific path
element has a path hook instantiated for it (based on the sys.path
element name) it is cached in sys.path_hooks_cache.
So, simply clearing out the path_hooks_cache entry for your main
library.zip file should cause the next import attempt to re-create a
new zipimporter instance and thus re-open the file and re-load the
directory information.
I don't know if py2exe installs the library.zip into sys.path just as
library.zip or with some path information, but try checking out the
keys in sys.path_hooks_cache from your application when it is running.
You should find an entry (probably the only one unless you explicitly
augment sys.path yourself) for library.zip - clear out that key after
the update and see how it works.
Heck, since you're the efficiency hit is likely not an issue, just
flush all of sys.path_hooks_cache and don't even worry about the
actual key name for library.zip. So a simple:
sys.path_importer_cache.clear()
call after your update completes may do the trick.
-- David
PS: In the same way that updating the library.zip under the running
application is tricky, you might run into issues if you end up trying
to update one of the extension modules. svn might not be able to
update it (depending on how it deals with in use files).
--
http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
In article [EMAIL PROTECTED], David Bolen [EMAIL PROTECTED] wrote: Are you perhaps trying to update the zip file in-place while it is still being used by the application? I'm not sure that's a safe operation. I'm sure it's not :) [lots of useful help snipped] the zipimport module has an attr called _zip_directory_cache, which is a dict you can .clear(). Still, reloading modules is hairy at best, its probably easiest to relaunch your app when the .zip file has changed. Just -- http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
Just wrote:
In article [EMAIL PROTECTED],
David Bolen [EMAIL PROTECTED] wrote:
Are you perhaps trying to update the zip file in-place while it is still
being used by the application? I'm not sure that's a safe operation.
I'm sure it's not :)
[lots of useful help snipped]
the zipimport module has an attr called _zip_directory_cache, which is a
dict you can .clear(). Still, reloading modules is hairy at best, its
probably easiest to relaunch your app when the .zip file has changed.
Just
what i do is as soon as the update is complete i close the app, but it
still gives the error, i tried clear() after update and before it, it
still got the same error. it's be nice to not have to fiddle around with
the zip file, i really think making py2exe create a dir instead of a zip
will be much better
here what i do anyway
if (os.name == 'nt') or (os.name == 'win32'):
client = pysvn.Client()
#get current revision number
CurrentRev = client.info('').revision.number
Check = client.update('')
sys.path_importer_cache.clear()
if Check.number CurrentRev:
self.Popup('Update installed, click ok and restart
','Update installed')
self.Destroy()
else:
InfoMsg.Update(3,'No Updates needed')
--
http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
Just [EMAIL PROTECTED] writes: the zipimport module has an attr called _zip_directory_cache, which is a dict you can .clear(). Still, reloading modules is hairy at best, its probably easiest to relaunch your app when the .zip file has changed. Except that he's getting an error during the process exit of the current execution, which is needed to restart. And if he updates to a different copy, there's the bootstrap problem of how to get it back into the standard location for the next restart since his application will need to have it to restart in the first place. -- David -- http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
Timothy Smith [EMAIL PROTECTED] writes:
what i do is as soon as the update is complete i close the app, but it
still gives the error, i tried clear() after update and before it, it
still got the same error. it's be nice to not have to fiddle around
with the zip file, i really think making py2exe create a dir instead
of a zip will be much better
Well, you'd still potentially have a problem if the update changed a
file in that directory that hadn't been imported yet, but now depended
on other updated files that your application had already loaded old
versions for. That's a general problem of updating modules beneath
the executing application, and not really specific to the zip file,
although you're getting a zip importer specific error related to that
in this case.
here what i do anyway
if (os.name == 'nt') or (os.name == 'win32'):
client = pysvn.Client()
#get current revision number
CurrentRev = client.info('').revision.number
Check = client.update('')
sys.path_importer_cache.clear()
if Check.number CurrentRev:
self.Popup('Update installed, click ok and restart
','Update installed')
self.Destroy()
else:
InfoMsg.Update(3,'No Updates needed')
Ah, it's more devious than I thought. Just pointed out the other
missing piece in his response.
Apparently there are two levels of caching that you've got to defeat
if you change the underlying zip:
1. A global file set of file directory cache information for any opened
zip file (for all files in the zip). This is held in the zipimport
module global _zip_directory_cache.
2. Individual file cached information within the zipimporter instance
that is kept in the path importer cache (sys.path_importer_cache).
Technically these are just references to the same individual entries
being held in the dictionary from (1).
So when you cleared out (2), it still found the cached directory at
the zipimport module level and re-used that information. But if you only
clear out (1), then the reference in (2) to the directory entries for
currently imported modules remains and still gets used.
I tried testing this with a small zip file that I first built with normal
compression on the entries, then imported one from a running interpreter,
and then rebuilt the zip without compression. I couldn't seem to get the
precise error you were getting, but doing this gave me a decompression
error upon an attempted reload of an imported module, since the cached
information still thought it was compressed.
After clearing both sys.path_importer_cache and
zipimport._zip_directory_cache, the reload went fine.
It's sort of unfortunate that you have to cheat with the private
cache clearing in this case. It might be worth an enhancement request
to see if zipimport could know to update itself if the timestamp on
the zip file changes, but this is sort of a very specialized scenario.
Although maybe just a public way to cleanly flush import cache
information would be useful.
-- David
--
http://mail.python.org/mailman/listinfo/python-list
Re: py2exe + svn - the final drama
David Bolen wrote:
Timothy Smith [EMAIL PROTECTED] writes:
what i do is as soon as the update is complete i close the app, but it
still gives the error, i tried clear() after update and before it, it
still got the same error. it's be nice to not have to fiddle around
with the zip file, i really think making py2exe create a dir instead
of a zip will be much better
Well, you'd still potentially have a problem if the update changed a
file in that directory that hadn't been imported yet, but now depended
on other updated files that your application had already loaded old
versions for. That's a general problem of updating modules beneath
the executing application, and not really specific to the zip file,
although you're getting a zip importer specific error related to that
in this case.
here what i do anyway
if (os.name == 'nt') or (os.name == 'win32'):
client = pysvn.Client()
#get current revision number
CurrentRev = client.info('').revision.number
Check = client.update('')
sys.path_importer_cache.clear()
if Check.number CurrentRev:
self.Popup('Update installed, click ok and restart
','Update installed')
self.Destroy()
else:
InfoMsg.Update(3,'No Updates needed')
Ah, it's more devious than I thought. Just pointed out the other
missing piece in his response.
Apparently there are two levels of caching that you've got to defeat
if you change the underlying zip:
1. A global file set of file directory cache information for any opened
zip file (for all files in the zip). This is held in the zipimport
module global _zip_directory_cache.
2. Individual file cached information within the zipimporter instance
that is kept in the path importer cache (sys.path_importer_cache).
Technically these are just references to the same individual entries
being held in the dictionary from (1).
So when you cleared out (2), it still found the cached directory at
the zipimport module level and re-used that information. But if you only
clear out (1), then the reference in (2) to the directory entries for
currently imported modules remains and still gets used.
I tried testing this with a small zip file that I first built with normal
compression on the entries, then imported one from a running interpreter,
and then rebuilt the zip without compression. I couldn't seem to get the
precise error you were getting, but doing this gave me a decompression
error upon an attempted reload of an imported module, since the cached
information still thought it was compressed.
After clearing both sys.path_importer_cache and
zipimport._zip_directory_cache, the reload went fine.
It's sort of unfortunate that you have to cheat with the private
cache clearing in this case. It might be worth an enhancement request
to see if zipimport could know to update itself if the timestamp on
the zip file changes, but this is sort of a very specialized scenario.
Although maybe just a public way to cleanly flush import cache
information would be useful.
-- David
awesome it looks like it's working now!
it's a very convenient why of keeping them all up today, now instead of
stuffing around making setup packages i can just run my makeexe.bat file
to create the py2exe files, svn commit -m new package and i'm done.
also there's no way for the staff to bugger it up ( well, it's fairly safe)
thanks very very much to everyone who made suggestions, this is what
makes OSS so good - the community input and support.
--
http://mail.python.org/mailman/listinfo/python-list
py2exe + svn - the final drama
ok, i am updating my program from my svn - this works, however, i am
getting the following error when i close the program. the zip file
successfully updates, so i'm curious as to the meaning of this.
Traceback (most recent call last):
File Main.pyo, line 820, in ValidateLogin
File Main.pyo, line 863, in ShowMainFrameItems
File glob.pyo, line 22, in glob
File glob.pyo, line 50, in glob1
File fnmatch.pyo, line 42, in filter
zipimport.ZipImportError: bad local file header in Z:\temp\library.zip
not that once i have finished client.update(''), it has successfully
updated the zipfile, i open a dialoge box saying click ok and restart
program AFTER i click on the above error pops up and my app shuts down
as intended.
ideas?
--
http://mail.python.org/mailman/listinfo/python-list
