Grrr, my 4th attempt at a reply without delivery ...
On Jul 8, 2008, at 6:57 PM, Alexey Tourbin wrote:
On Mon, Jul 07, 2008 at 11:44:07PM +0200, Jeff Johnson wrote:
- make gzdio.c standalone.
BTW, I have rsyncable gzdio implemntation (this does not require
patched zlib, one only has to call gzflush() at certain sync points).
I'd like to not carry internal zlib.
You are absolutely correct that the --rsyncable issue is calling
gzflush() at
certain sync points.
It is known to work well. Please review the patches and tell me
whether you want it or not.
http://git.altlinux.org/people/at/packages/rpm.git?
a=commitdiff;h=c761902b
http://git.altlinux.org/people/at/packages/rpm.git?
a=commitdiff;h=f7b5ee1e
http://git.altlinux.org/people/at/packages/rpm.git?
a=commitdiff;h=8d5e355e
http://git.altlinux.org/people/at/packages/rpm.git?
a=commitdiff;h=52b2499a
OTOH, here are other fundamental issues that are relevant:
0) No one understands why --rsyncable is important, or why gzip != zlib,
or why the "fuzzy" name patch in rsync would be a tremendous bandwidth
saving for *.rpm packages. I've been tracking the issue for like 6+
years,
and what is fundamentally needed is a very clear demonstration,
including
publicized benchmarks and likely a drop-in "production" ready transport
implementation, for any --rsyncable code to be worth the effort. JMHO
based on 6+ years of explaining ...
So "Using unpatched zlib external!" is the wrong reason to justify
the changes.
One needs to make the fundamental reasons for the changes more
obvious to
end lusers.
1) Your changes add 2-3 new wrapper layers (at least cpio/rsync) to
an already complex code base (with stacked I/O handlers accessed
through libio,
and with emulated libio for non-glibc portability).
I personally dunno whether I can debug rpm problems if your
additional patches are added.
(OTOH, I have no problems at all with an alternative gzdio
implementation,
or with #ifdef'd additions, just not the main "production" RPM code
path please).
2) From a fundamental coding design POV, adding --rsyncable to rpmio
code is just
plain wrong. That's as true for the patched internal zlib as it is
for your gzdio
patches. The issue of when gzflush() is called is fundamentally a
compression
and rsync transport, not a *.rpm payload packaging, issue.
So isn't a pure API/ABI drop-in replacement for gzio(3), with a
"gzwrite" rather
than a "rsyncable_gzwrite" symbol a better approach? There are many
applications,
not just rpm, that _MUST_ participate in an efficient *.rpm
distribution framework,
so patching gzdio in rpmio is just a tiny piece of a much bigger
puzzle, where "gzwrite"
rather than "rsyncable_gzwrite" is likelier to be successful. JMHO,
YMMV, as always.
3) As sole developer/designer of RPMIO, I also believe that my
choice to use the
gzopen(3) API was a design brain fart. The gzopen(3) API is/was
appealingly simple
at the time (1999), but increasingly is becoming _THE_ roadblock to
adding additional
functionality, like XAR and network transport. to RPMIO.
This code (from rpm2cpio.c an insanely simple program) breaks if
handlers, rather than
plain old integer fdno's, are used with RPMIO:
gzdi = Fdopen(fdi, rpmio_flags); /* XXX gzdi == fdi */
if (gzdi == NULL) {
fprintf(stderr, _("cannot re-open payload: %s\n"), Fstrerror
(gzdi));
exit(EXIT_FAILURE);
}
Note that neon and XAR handlers are not based on fdno's, rather
opaque pointers, and so
interoperate badly with gzdio/bzdio/lzdio pointer handlers. There's
no provision
for say, pushing neon under a compression library, and RPMIO <-> XAR
have
dueling compression stacks.
Which is why I call the choice of using gzopen(3) a "brain fart", a
compression layer
doesn't need heavt gzopen(3) I/O methods, only buffer compress/
decompress methods.
(aside) See XAR and rsync implementations using gzip/bzip2/lzma to
see more precisely
what I think RPMIO needs to do instead of using the gzopen(3) API.
In summary, your patches are quite nicely done and extremely clever,
but I'm not
sure that RPMIO could/should use them, as I've tried to describe above.
I'm most certainly willing to be convinced otherwise however.
73 de Jeff
______________________________________________________________________
RPM Package Manager http://rpm5.org
Developer Communication List rpm-devel@rpm5.org
