Bug#739411: udpkg: Support for data.tar, control.tar and control.tar.xz
tag -1 + pending thanks Hi, On Tue, Feb 18, 2014 at 01:01:48PM +0100, Guillem Jover wrote: Here's a patch series adding data.tar, control.tar and control.tar.xz support, the uncompressed member support just out of completeness. To create a deb package with a different control.tar compression you can use «dpkg-deb --uniform-compression pkg/» for example. Note I've only tested «udpkg -c» and «udpkg -f», the changes in unpacking are pretty obvious, but testing them would be nice. I've also used cat for the uncompressed members as I didn't think further optimization there was worth it(?). I've also tried to use the prevailing coding-style, which in some cases conflicted with the immediate surrounding code. given KiBi's analysis there's no immediate rush to go all-xz for jessie, but we should still land the patches to udpkg, of course. As for the cat: You went from ar -p %s %s%s|%s -c|tar -x to ar -p %s %s%s|%s|tar -x. I guess you could have moved the pipe into the output of decompression_tool as well and do ar -p %s %s%s%s|tar -x with |gunzip -c, |unxz -c, and . But then I think your approach is clear to readers, so that's good. ;-) I pushed your patches to the repository. Very much thanks for them. Kind regards Philipp Kern signature.asc Description: Digital signature
Bug#739411: udpkg: Support for data.tar, control.tar and control.tar.xz
Package: udpkg Version: 1.16 Severity: wishlist Tags: patch X-Debbugs-CC: Philipp Kern pk...@debian.org Hi! Here's a patch series adding data.tar, control.tar and control.tar.xz support, the uncompressed member support just out of completeness. To create a deb package with a different control.tar compression you can use «dpkg-deb --uniform-compression pkg/» for example. Note I've only tested «udpkg -c» and «udpkg -f», the changes in unpacking are pretty obvious, but testing them would be nice. I've also used cat for the uncompressed members as I didn't think further optimization there was worth it(?). I've also tried to use the prevailing coding-style, which in some cases conflicted with the immediate surrounding code. Thanks, Guillem -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#739411: udpkg: Support for data.tar, control.tar and control.tar.xz
On Tue, 2014-02-18 at 13:01:48 +0100, Guillem Jover wrote:
Package: udpkg
Version: 1.16
Severity: wishlist
Tags: patch
X-Debbugs-CC: Philipp Kern pk...@debian.org
Here's a patch series adding data.tar, control.tar and control.tar.xz
support, the uncompressed member support just out of completeness. To
create a deb package with a different control.tar compression you can
use «dpkg-deb --uniform-compression pkg/» for example.
Note I've only tested «udpkg -c» and «udpkg -f», the changes in unpacking
are pretty obvious, but testing them would be nice. I've also used cat
for the uncompressed members as I didn't think further optimization there
was worth it(?). I've also tried to use the prevailing coding-style,
which in some cases conflicted with the immediate surrounding code.
And then forgot to attach the patches…
Thanks,
Guillem
From b806fc9a73e79b561fad9e40e34d44d86031e4a1 Mon Sep 17 00:00:00 2001
From: Guillem Jover guil...@debian.org
Date: Sun, 16 Feb 2014 02:25:10 +0100
Subject: [PATCH 1/4] Pass member_base as an argument to get_compression_type()
Allow other member base names, instead of hardcoding data.tar.
---
udpkg.c | 16 +---
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/udpkg.c b/udpkg.c
index 2adfd16..73fa9d8 100644
--- a/udpkg.c
+++ b/udpkg.c
@@ -163,7 +163,9 @@ static const char *decompression_tool (const compression_type t) {
}
}
-static compression_type get_compression_type (struct package_t *pkg) {
+static compression_type get_compression_type(struct package_t *pkg,
+ const char *member_base)
+{
FILE *infp = NULL;
char buf[1024];
char *extension = NULL;
@@ -177,8 +179,8 @@ static compression_type get_compression_type (struct package_t *pkg) {
}
while (fgets(buf, sizeof(buf), infp)) {
- if (strncmp(buf, data_member_base, strlen(data_member_base)) == 0) {
- extension = buf + strlen(data_member_base);
+ if (strncmp(buf, member_base, strlen(member_base)) == 0) {
+ extension = buf + strlen(member_base);
if (extension[strlen(extension) - 1] == '\n')
extension[strlen(extension) - 1] = '\0';
break;
@@ -188,7 +190,7 @@ static compression_type get_compression_type (struct package_t *pkg) {
if (extension == NULL) {
FPRINTF(stderr, No %s* found in %s\n,
- data_member_base, pkg-file);
+ member_base, pkg-file);
return unknown_compression;
}
@@ -200,7 +202,7 @@ static compression_type get_compression_type (struct package_t *pkg) {
}
else {
FPRINTF(stderr, Invalid compression type for %s* of %s\n,
- data_member_base, pkg-file);
+ member_base, pkg-file);
return unknown_compression;
}
}
@@ -226,7 +228,7 @@ static int dpkg_dounpack(struct package_t *pkg)
DPRINTF(Unpacking %s\n, pkg-package);
- compression_type = get_compression_type(pkg);
+ compression_type = get_compression_type(pkg, data_member_base);
if (compression_type == unknown_compression)
return 1;
@@ -556,7 +558,7 @@ static int dpkg_contents(struct package_t *pkg)
reqarg(pkg);
- compression_type = get_compression_type(pkg);
+ compression_type = get_compression_type(pkg, data_member_base);
if (compression_type == unknown_compression)
return 1;
--
1.9.0.rc3.244.g3497008
From 193c1b896291da42844953bdf693245ec4c8adaa Mon Sep 17 00:00:00 2001
From: Guillem Jover guil...@debian.org
Date: Sun, 16 Feb 2014 02:28:34 +0100
Subject: [PATCH 2/4] Add support for uncompressed deb members
---
udpkg.c | 21 ++---
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/udpkg.c b/udpkg.c
index 73fa9d8..ef6bdf7 100644
--- a/udpkg.c
+++ b/udpkg.c
@@ -18,7 +18,7 @@
static int force_configure = 0;
static int loadtemplate = 1;
-static const char *data_member_base = data.tar.;
+static const char *data_member_base = data.tar;
/*
* Main udpkg implementation routines
@@ -144,21 +144,24 @@ typedef enum compression_type compression_type;
enum compression_type {
gz_compression,
xz_compression,
+ no_compression,
unknown_compression,
};
static const char *compression_extension (const compression_type t) {
switch (t) {
- case gz_compression: return gz;
- case xz_compression: return xz;
+ case gz_compression: return .gz;
+ case xz_compression: return .xz;
+ case no_compression: return ;
default: return ;
}
}
static const char *decompression_tool (const compression_type t) {
switch (t) {
- case gz_compression: return gunzip;
- case xz_compression: return unxz;
+ case gz_compression: return gunzip -c;
+ case xz_compression: return unxz -c;
+ case no_compression: return cat;
default: return ;
}
}
@@ -200,6 +203,10 @@ static compression_type get_compression_type(struct package_t *pkg,
else if (strcmp(extension, compression_extension(xz_compression)) == 0) {
return xz_compression;
}
+ else if (strcmp(extension, compression_extension(no_compression)) == 0)
+ {
+ return no_compression;
+ }
else {
FPRINTF(stderr, Invalid compression type for %s* of %s\n,
Bug#739411: udpkg: Support for data.tar, control.tar and control.tar.xz
Guillem Jover guil...@debian.org (2014-02-18): Here's a patch series adding data.tar, control.tar and control.tar.xz support, the uncompressed member support just out of completeness. To create a deb package with a different control.tar compression you can use «dpkg-deb --uniform-compression pkg/» for example. So I've played a little using deb-reversion to generate local versions (see #739437), and using that loop (once with xz, once with none): for i in *.udeb; do dpkg -x $i foo dpkg --control $i foo/DEBIAN fakeroot dpkg-deb -b -Znone --uniform-compression foo dpkg-name -o foo.deb rm -r foo; done # .deb → .udeb renaming # reprepro includeudeb ... # boot with debian-installer/allow_unauthenticated=true I haven't performed a full installation, just checked that with unpatched udpkg I'm getting tar magic/read failures, and success with a new udpkg, at least until the partitioning step. Note I've only tested «udpkg -c» and «udpkg -f», the changes in unpacking are pretty obvious, but testing them would be nice. I've also used cat for the uncompressed members as I didn't think further optimization there was worth it(?). cat is probably OK. I'm not sure there's any need or interest in having uncompressed udebs anyway… Mraw, KiBi. signature.asc Description: Digital signature
Bug#739411: udpkg: Support for data.tar, control.tar and control.tar.xz
Cyril Brulebois k...@debian.org (2014-02-18): So I've played a little using deb-reversion to generate local versions (see #739437), and using that loop (once with xz, once with none): for i in *.udeb; do dpkg -x $i foo dpkg --control $i foo/DEBIAN fakeroot dpkg-deb -b -Znone --uniform-compression foo dpkg-name -o foo.deb rm -r foo; done # .deb → .udeb renaming # reprepro includeudeb ... # boot with debian-installer/allow_unauthenticated=true Iterating over the Filename's in jessie/main/debian-installer/binary-amd64/Packages.gz and recompressing every udeb with -Zxz --uniform-compression, I'm only getting an overall 5% gain in udeb size. For the curious I'm attaching a little PDF file with per-udeb delta. Before: 47163752 After: 44718700 So it looks to me there's no need to rush for xz-only udebs. Especially since stable's d-i might be used to install testing, and also because we're going to add a little bit of entropy with wheezy-backports images (even though it should be trivial to pull udpkg from backports so that it can deal with any xz+xz udeb it comes across). Mraw, KiBi. signature.asc Description: Digital signature
