Hey guys,
I have some pretty experimental code in trunk which I'd love for you to
try out.
In general, it is an attempt to update InstallMgr to prefer HTTPS zip
package downloads if HTTPS is a supported protocol on the platform and
they exists on the remote source. Details follow:
1) new unzip expansion support in the engine:
we've wanted, for a while now, to support zip archive expansion;
historically, we've supported tar.gz expansion. My issue in the past is
that zlib only supports the decompression action, not actually reading
an archive of files and expanding them. zlib does include a contrib/
folder which has enough functionality to provide this. I've finally
waded through it all and it appears it might be cross-platform enough
for us to compile on our targets. The new files are:
src/modules/common/ioapi.c
src/modules/common/unzip.c
include/internal/unzip/ioapi.h
include/internal/unzip/unzip.h
with a new static method on our class ZipCompress::unZip(const char
*sourceZipPath, const char *destPath);
I've done my best to try to add the proper lines to our cmake system.
I've only tested our autotools build system (autotools users, be sure to
./autogen.sh and ./usrinst.sh to refresh and pick up the new files)
2) InstallMgr will now look for a cached copy of the module at
[Source]/packages/ModName.zip
Just like InstallMgr has looked for a cached copy of mods.d.tar.gz at
the source root, it now looks for a cached copy of the moduled zipped up
under packages/. If it finds the zip file, it will download that and
unzip instead of trying to download file by file.
To test, I've added a packages/ symlink for the CrossWire repo pointing
to our JSword zip bundles. You can see the packages folder here:
https://crosswire.org/ftpmirror/pub/sword/raw/
The idea is that any repo which wants to make these package zips
available (many do anyway because JSword has always required it) can now
place them in a standard location at the root of their repo under a
folder called packages/
3) InstallMgr now supports a concept called "chained sources"
This allows a repo to specify multiple transport configurations (e.g.,
FTP and HTTP). Sources are chained if multiple configuration entries
exist for the same repo name. This isn't fully implemented yes, but it
is finished enough to support the next item, #4, below.
4) InstallMgr now supports a new configuration line in the [Repos]
section HTTPSPackagePreference.
This allows a repo to say that it prefers HTTPS Packages, if possible.
This will cause an HTTPS install source to be created as the first
attempted installation route, but only for caches packages, if the
platform doesn't have HTTPS support, or the cached package doesn't
exist, it will move on to the next chained source (e.g., fall back to FTP).
To test, I've added an entry for CrossWire which you can see in our
master repo list:
https://crosswire.org/ftpmirror/pub/sword/masterRepoList.conf
5) InstallMgr now first tries to sync its configuration from our master
repo source list via HTTPS and then falls back to FTP if unsuccessful.
This all let's us use our HTTP remote transport we've had in the engine
for a while, but avoids the problem of parsing various http folder
listing formats. It also speeds up FTP downloads if the cached zip
archive file exists on the server.
The new HTTPSPackagePreference configuration line will be ignored by old
SWORD software, and if the packages/ folder with zips doesn't exist on
the remote source, all falls back to the previous method of installation.
The bottom line is that I believe one can now:
cd sword/utilities/
./installmgr -sc
./installmgr -r CrossWire
./installmgr -ri CrossWire KJV
completely over HTTPS and without downloading individual files.
Let me know if you have a chance to try it out.
Happy Memorial Day! Our Lord's best for you,
Troy
_______________________________________________
sword-devel mailing list: sword-devel@crosswire.org
http://crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
