Re: Reimplement mksplit.pl in C
[ I subscribed to the list, thanks for the Cc:'s ]
On Sat, Apr 03, 2010 at 12:20:52PM -0600, Marcelo E. Magallon wrote:
The two cases I can think of are splitting an archive that
splits to a single piece, splitting to multiple pieces and
some failure cases.
Right now I do something like:
cat test.deb |
build-tree/dpkg-split/mksplit test.deb 1024 test 4608 2048 no
cp test.deb test.deb.orig
dpkg-split -j test*.deb
cmp test.deb test.deb.orig echo OK
but not much more.
I did some refactoring, and reused a couple of bits of libdpkg.
I was hoping for libdpkg to have something like:
package = get_deb_field(my_package.deb, Package);
but I didn't find it. Did I miss it? Is there something close
to that?
Anyways, the updated patches are attached.
Marcelo
From 40cbd26e938f3ffda58803c6e25d6e4c9a836b96 Mon Sep 17 00:00:00 2001
From: Marcelo E. Magallon marcelo.magal...@gmail.com
Date: Fri, 2 Apr 2010 22:27:50 -0600
Subject: [PATCH 1/5] Reimplement mksplit in C
My original intention was to do without some of the pipes, but for the
initial implementation this is just a translation of the original Perl
program into C, without much in the way of optimization. It produces
the same output and exits in the same way as the original program (mod
exit status probably).
The reimplementation can use some refactoring love.
---
dpkg-split/Makefile.am | 12 ++-
dpkg-split/mksplit.c | 279
2 files changed, 288 insertions(+), 3 deletions(-)
create mode 100644 dpkg-split/mksplit.c
diff --git a/dpkg-split/Makefile.am b/dpkg-split/Makefile.am
index 6f6043c..3ea9d00 100644
--- a/dpkg-split/Makefile.am
+++ b/dpkg-split/Makefile.am
@@ -24,10 +24,16 @@ dpkg_split_LDADD = \
../lib/compat/libcompat.a \
$(LIBINTL)
+mksplit_SOURCES = \
+ mksplit.c
-pkglib_SCRIPTS = mksplit
-EXTRA_DIST = mksplit.pl
-CLEANFILES = $(pkglib_SCRIPTS)
+mksplit_LDADD = \
+ ../lib/dpkg/libdpkg.a \
+ ../lib/compat/libcompat.a \
+ $(LIBINTL)
+
+pkglib_PROGRAMS = mksplit
+CLEANFILES = $(pkglib_PROGRAMS)
do_perl_subst = $(AM_V_GEN) sed -e s:^\#![:space:]*/usr/bin/perl:\#!$(PERL):
diff --git a/dpkg-split/mksplit.c b/dpkg-split/mksplit.c
new file mode 100644
index 000..5ca34d6
--- /dev/null
+++ b/dpkg-split/mksplit.c
@@ -0,0 +1,279 @@
+/* This program is only supposed to be called by dpkg-split.
+ * Its arguments are:
+ * sourcefile partsize prefix totalsize partsizeallow msdostruncyesno
+ * Stdin is also redirected from the source archive by dpkg-split.
+ *
+ * Copyright © 1995 Ian Jackson i...@chiark.greenend.org.uk
+ * Copyright © 2010 Marcelo E. Magallon mmaga...@debian.org
+ *
+ * This is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/.
+ */
+
+#include stdio.h
+#include stdlib.h
+#include time.h
+#include limits.h
+#include error.h
+#include string.h
+#include ctype.h
+#include errno.h
+#include unistd.h
+
+#define MSDOS_PATH_MAX 10
+
+void checked_free(void *ptr)
+{
+if (ptr) free(ptr);
+}
+
+void add(FILE *stream, char* id, char* data, size_t len)
+{
+time_t current_time;
+
+current_time = time(NULL);
+fprintf(stream, %-16s%-12d0 0 100644 %-10d%c\n,
+id, (int)current_time, (int)len, 0140);
+fwrite(data, 1, len, stream);
+fprintf(stream, %s, (len 1) ? \n : );
+}
+
+char* output(char *cmd)
+{
+FILE * file = NULL;
+int len = getpagesize();
+char *result = (char *)malloc(len);
+char *p = result;
+
+if (result == NULL)
+goto error_out;
+
+if ((file = popen(cmd, r)) == NULL)
+goto error_out;
+
+while (len 0 !feof(file))
+{
+int n;
+n = fread(p, 1, len, file);
+if (ferror(file))
+goto error_out;
+p += n;
+len -= n;
+}
+goto out;
+
+error_out:
+if (result)
+{
+free(result);
+result = NULL;
+}
+
+out:
+if (file)
+pclose(file);
+return result;
+}
+
+char* get_field(char *filename, char *field)
+{
+char *result;
+char cmd[PATH_MAX];
+sprintf(cmd, dpkg-deb --field %s %s, filename, field);
+
+result = output(cmd);
+if (result)
+{
+char *p;
+for (p = result + strlen(result) - 1; p = result isspace(*p); --p)
+*p = 0;
+}
+
+return result;
+}
+
+char* get_md5sum(char *filename)
+{
+char *result;
+char cmd[PATH_MAX
Re: Reimplement mksplit.pl in C
On Sat, Apr 03, 2010 at 09:23:54PM +0200, Raphael Hertzog wrote: The more recent reference is in http://wiki.debian.org/Teams/Dpkg/RoadMap Thanks! Marcelo -- To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100405034717.gb21...@esk
Re: Reimplement mksplit.pl in C
On Sat, Apr 03, 2010 at 01:19:33AM -0500, Jonathan Nieder wrote: I could only find approx one reference for this goal [1]. Ah, yes, that was it, thanks! Removing Perl from the essential set. While going to bed, I actually figured that was the reason why I wasn't finding perl in the dependencies, but I didn't remember this was the reason I had read about for the reimplementation. Luckily, even if that doesn’t pan out, there are plenty of other benefits to well written C code. Most importantly, it makes code sharing a little easier. Sure. I'd like to take this implementation and factor out whatever is reusable. There are a couple of maybe-useful gimmicks in there, for example the Perl idiom `program` transliterated into C. I cheated and took a guess regarding how large should a buffer be to be large enough for this use case: my guess is one page. There are other hard-coded buffer length limits, too. Memory concious it ain't right now, this code. I guess this would also make repairing a broken perl installation a little simpler on some system with only access to small media, though I haven’t encountered a need for dpkg-split in a long while. Thanks for working on it. Same here. The last time I used dpkg-split for something was back in ... 1996? I did compare the output with the Perl version in several test cases and it's identical. Some test cases (or methods for producing them) would make me very happy. dpkg doesn’t have enough automated tests of basic functionality. Automake provides a rudimentary test harness [2]. I was thinking about that. The problem with mksplit is that the generated files contain an embedded timestamp, so when I say identical I actually mean identical modulo the embedded timestamp. I mean, you can't just cmp two sets of files. Beyond that, mksplit has several bits of code identifiable as functions that ought to be separated and those could have unit tests. I would have to look in more detail at dpkg-split to see if it's possible to devise a good automated test for this functionality. The two cases I can think of are splitting an archive that splits to a single piece, splitting to multiple pieces and some failure cases. There's a check in mksplit.pl (also implemented in mksplit.c) that I'm still trying to figure out how to trigger (the corresponding error message refers to the header being too large). I think this should be bin_PROGRAMS = dpkg-split pkglib_PROGRAMS = mksplit so that mksplit stays in /usr/lib/dpkg. Ah, right, thanks, updated patch attached. Marcelo From acbd7b59d0d48ec47191d67846029aedc3c49afe Mon Sep 17 00:00:00 2001 From: Marcelo E. Magallon marcelo.magal...@gmail.com Date: Fri, 2 Apr 2010 22:27:50 -0600 Subject: [PATCH] Reimplement mksplit in C My original intention was to do without some of the pipes, but for the initial implementation this is just a translation of the original Perl program into C, without much in the way of optimization. It produces the same output and exits in the same way as the original program (mod exit status probably). The reimplementation can use some refactoring love. --- dpkg-split/Makefile.am | 12 ++- dpkg-split/mksplit.c | 279 2 files changed, 288 insertions(+), 3 deletions(-) create mode 100644 dpkg-split/mksplit.c diff --git a/dpkg-split/Makefile.am b/dpkg-split/Makefile.am index 6f6043c..3ea9d00 100644 --- a/dpkg-split/Makefile.am +++ b/dpkg-split/Makefile.am @@ -24,10 +24,16 @@ dpkg_split_LDADD = \ ../lib/compat/libcompat.a \ $(LIBINTL) +mksplit_SOURCES = \ + mksplit.c -pkglib_SCRIPTS = mksplit -EXTRA_DIST = mksplit.pl -CLEANFILES = $(pkglib_SCRIPTS) +mksplit_LDADD = \ + ../lib/dpkg/libdpkg.a \ + ../lib/compat/libcompat.a \ + $(LIBINTL) + +pkglib_PROGRAMS = mksplit +CLEANFILES = $(pkglib_PROGRAMS) do_perl_subst = $(AM_V_GEN) sed -e s:^\#![:space:]*/usr/bin/perl:\#!$(PERL): diff --git a/dpkg-split/mksplit.c b/dpkg-split/mksplit.c new file mode 100644 index 000..5ca34d6 --- /dev/null +++ b/dpkg-split/mksplit.c @@ -0,0 +1,279 @@ +/* This program is only supposed to be called by dpkg-split. + * Its arguments are: + * sourcefile partsize prefix totalsize partsizeallow msdostruncyesno + * Stdin is also redirected from the source archive by dpkg-split. + * + * Copyright © 1995 Ian Jackson i...@chiark.greenend.org.uk + * Copyright © 2010 Marcelo E. Magallon mmaga...@debian.org + * + * This is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You
Reimplement mksplit.pl in C
Hi,
somewhere I read that a reimplementation of mksplit.pl in C was
needed in order to remove a dependency of dpkg on Perl, but I
can't find the reference anymore. Oddly enough I don't see a
dependency of dpkg on perl either:
Package: dpkg
Version: 1.15.6.1
Replaces: manpages-de (= 0.4-3), manpages-pl (= 20051117-1)
Pre-Depends: libbz2-1.0, libc6 (= 2.3), libselinux1 (= 1.32), zlib1g (=
1:1.1.4), coreutils (= 5.93-1), xz-utils
Suggests: apt
Conflicts: apt ( 0.7.7), aptitude ( 0.4.7-1), dpkg-dev ( 1.14.16),
dpkg-iasearch ( 0.11), sysvinit ( 2.82-1)
Breaks: emacs21 ( 21.4a+1-5.7), emacs21-nox ( 21.4a+1-5.7), emacs22 (=
22.3+1-1), emacs22-gtk (= 22.3+1-1), emacs22-nox (= 22.3+1-1), jed (
1:0.99.18+dfsg.1-13), jed-extra (= 2.5.3-2), konqueror (= 4:4.2.96-1), pinfo
( 0.6.9-3.1), tkinfo ( 2.8-3.1), xemacs21-support ( 21.4.22-2), xjed (
1:0.99.18+dfsg.1-13)
Anyways, attached is a patch to do just that. It's implemented
against the master Git branch. As indicated in the patch, this
is a straight translation and it would like to have some
refactoring love. I did compare the output with the Perl
version in several test cases and it's identical.
Please Cc: me, I'm not subscribed.
Cheers,
Marcelo
From 267685fb5750a47b536ab817a72597836abe5c5f Mon Sep 17 00:00:00 2001
From: Marcelo E. Magallon mmaga...@debian.org
Date: Fri, 2 Apr 2010 22:27:50 -0600
Subject: [PATCH] Reimplement mksplit in C
My original intention was to do without some of the pipes, but for the
initial implementation this is just a translation of the original Perl
program into C, without much in the way of optimization. It produces
the same output and exits in the same way as the original program (mod
exit status probably).
The reimplementation can use some refactoring love.
---
dpkg-split/Makefile.am | 10 ++-
dpkg-split/mksplit.c | 279
2 files changed, 287 insertions(+), 2 deletions(-)
create mode 100644 dpkg-split/mksplit.c
diff --git a/dpkg-split/Makefile.am b/dpkg-split/Makefile.am
index 6f6043c..c43025d 100644
--- a/dpkg-split/Makefile.am
+++ b/dpkg-split/Makefile.am
@@ -9,7 +9,7 @@ AM_CPPFLAGS = \
-I$(top_srcdir)/lib
-bin_PROGRAMS = dpkg-split
+bin_PROGRAMS = dpkg-split mksplit
dpkg_split_SOURCES = \
dpkg-split.h \
@@ -24,9 +24,15 @@ dpkg_split_LDADD = \
../lib/compat/libcompat.a \
$(LIBINTL)
+mksplit_SOURCES = \
+ mksplit.c
+
+mksplit_LDADD = \
+ ../lib/dpkg/libdpkg.a \
+ ../lib/compat/libcompat.a \
+ $(LIBINTL)
pkglib_SCRIPTS = mksplit
-EXTRA_DIST = mksplit.pl
CLEANFILES = $(pkglib_SCRIPTS)
diff --git a/dpkg-split/mksplit.c b/dpkg-split/mksplit.c
new file mode 100644
index 000..5ca34d6
--- /dev/null
+++ b/dpkg-split/mksplit.c
@@ -0,0 +1,279 @@
+/* This program is only supposed to be called by dpkg-split.
+ * Its arguments are:
+ * sourcefile partsize prefix totalsize partsizeallow msdostruncyesno
+ * Stdin is also redirected from the source archive by dpkg-split.
+ *
+ * Copyright © 1995 Ian Jackson i...@chiark.greenend.org.uk
+ * Copyright © 2010 Marcelo E. Magallon mmaga...@debian.org
+ *
+ * This is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/.
+ */
+
+#include stdio.h
+#include stdlib.h
+#include time.h
+#include limits.h
+#include error.h
+#include string.h
+#include ctype.h
+#include errno.h
+#include unistd.h
+
+#define MSDOS_PATH_MAX 10
+
+void checked_free(void *ptr)
+{
+if (ptr) free(ptr);
+}
+
+void add(FILE *stream, char* id, char* data, size_t len)
+{
+time_t current_time;
+
+current_time = time(NULL);
+fprintf(stream, %-16s%-12d0 0 100644 %-10d%c\n,
+id, (int)current_time, (int)len, 0140);
+fwrite(data, 1, len, stream);
+fprintf(stream, %s, (len 1) ? \n : );
+}
+
+char* output(char *cmd)
+{
+FILE * file = NULL;
+int len = getpagesize();
+char *result = (char *)malloc(len);
+char *p = result;
+
+if (result == NULL)
+goto error_out;
+
+if ((file = popen(cmd, r)) == NULL)
+goto error_out;
+
+while (len 0 !feof(file))
+{
+int n;
+n = fread(p, 1, len, file);
+if (ferror(file))
+goto error_out;
+p += n;
+len -= n;
+}
+goto out;
+
+error_out:
+if (result)
+{
+free(result);
+result = NULL;
+}
+
+out:
+if (file)
+pclose(file);
+return result;
+}
+
+char
Re: dpkg on other unix
On Thu, Dec 10, 1998 at 07:42:28AM -0500, Ben Collins wrote: How much interest is there in having some 'ports' of debian that are only meant to be overlays to other OS's such as Solaris? I think Solaris would be the only one I am interested in. How do update-rc.d, start-stop-daemon, update-alternatives and install-info play with Solaris? Or did I misunderstood you? Is your intent to use only dpkg to manage local packages or use dpkg to install debian packages as well? Marcelo
