bug#6789: propose renaming gnulib memxfrm to amemxfrm (naming collision with coreutils)
Paul Eggert egg...@cs.ucla.edu writes: On 08/06/10 01:22, Paolo Bonzini wrote: ISAAC is a RNG, so wouldn't that have the same problem above? You definitely need to use a hash function, it's just that you do not need a cryptographic one. I had been thinking of using ISAAC by making the key its seed, and asking it to generate some random values, and then comparing the random values. Any RNG can be used (or abused :-) in this way. I just now tried, that, though, and discovered that on my million line benchmark the MD5 method is about 4 times faster than the ISAAC-based method. So that idea was not a good one. I suppose we could try a non-cryptographic hash function at some point. I suspect FNV or Xorshift would be faster, since they are so simple: http://en.wikipedia.org/wiki/Fowler_Noll_Vo_hash http://en.wikipedia.org/wiki/Xorshift /Simon
bug#6789: propose renaming gnulib memxfrm to amemxfrm (naming collision with coreutils)
Paul Eggert egg...@cs.ucla.edu writes: Come to think of it, looking at gnulib memxfrm gave me an idea to improve the performance of GNU sort by bypassing the need for an memxfrm-like function entirely. I pushed a patch to do that at http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=2b49b140cc13cf36ec5ee5acaca5ac7bfeed6366. I don't know this code at all, but would your approach lead to problems if two different strings have the same MD5 hash? MD5 is broken, and finding collisions takes just seconds on normal PC. See: http://en.wikipedia.org/wiki/MD5#Security /Simon
Re: [PATCH] issue warnings for more missing optional libraries
Bruno Haible br...@clisp.org writes:
I like it. Even more I like the configure files that print a summary of the
most important decisions or findings at the end of the configuration, thus
giving an opportunity to the user to re-configure with different --with or
--enable options. (Given the size of the configure output, hardly anyone looks
through it in its entirety.)
I like this too. In libidn it looks like this:
version: 1.16 shared 16:45:5
Host type:i586-pc-mingw32msvc
Install prefix: /home/builder/mingw
Compiler: i586-mingw32msvc-gcc -std=gnu99
Warning flags:
Library types:Shared=yes, Static=yes
Valgrind:
Version script: yes
TLD code: yes
Build Java port: no
Build C# port:no
Which is achieved by this code:
AC_MSG_NOTICE([summary of build options:
version: ${VERSION} shared $LT_CURRENT:$LT_REVISION:$LT_AGE
Host type:${host}
Install prefix: ${prefix}
Compiler: ${CC}
Warning flags:${WARN_CFLAGS}
Library types:Shared=${enable_shared}, Static=${enable_static}
Valgrind: ${VALGRIND}
Version script: $have_ld_version_script
TLD code: $enable_tld
Build Java port: $enable_java
Build C# port:$enable_csharp
])
/Simon
Re: new snapshot available: coreutils-7.6.63-addb6
Pádraig Brady p...@draigbrady.com writes: Jim Meyering wrote: Pádraig Brady wrote: Jim Meyering wrote: Eric Blake wrote: According to Pádraig Brady on 10/5/2009 3:53 PM: This is a new test, but FC5 is s old, that I'm not sure it's worth worrying about. March 2006? The failure is probably a function of the kernel. Which is it? In summary this is what fails: $ touch a $ ln -s a symlink $ ln -L symlink hardlink ln: creating hard link `hardlink' = `symlink': Invalid argument `man linkat` says that AT_SYMLINK_FOLLOW is only supported since 2.6.18 and my FC5 system is 2.6.17 This should fix it. I don't have access to FC5, but I tested the new code path by priming the cache (gl_cv_func_linkat_follow=runtime ./configure) along with a temporary setting of have_follow_really=-1 in linkat.c. I also verified that the replacement is not picked up on cygwin 1.7, where AT_SYMLINK_FOLLOW was implemented at the same time as linkat. The patch copies from areadlink.c, as well as link_follow earlier in linkat.c, to create two new fd-relative helpers. For now, I didn't see any reason to expose them, but areadlinkat may someday be worth making into a full-blown module. Wow, that was quick. Thanks. I should have read this first. I was just reviewing the changes in gnulib and see a few that should be included in the imminent coreutils beta release, so will probably take this one, too. Needs a couple of tweaks.. This needs to be added to linkat.c (seems like it should be refactored somewhere?) #ifndef SIZE_MAX # define SIZE_MAX ((size_t) -1) #endif #ifndef SSIZE_MAX # define SSIZE_MAX ((ssize_t) (SIZE_MAX / 2)) #endif This should do it: From 6f6420cc9705dcfa545a28c674fddf5703e72c86 Mon Sep 17 00:00:00 2001 From: Jim Meyering meyer...@redhat.com Date: Tue, 6 Oct 2009 11:11:39 +0200 Subject: [PATCH] linkat: avoid compilation failure * lib/linkat.c: Include stdint.h for use of SIZE_MAX. That works thanks. I suppose these should include stdint.h also? areadlink.c:# define SIZE_MAX ((size_t) -1) areadlink-with-size.c:# define SIZE_MAX ((size_t) -1) backupfile.c:# define SIZE_MAX ((size_t) -1) fnmatch.c:# define SIZE_MAX ((size_t) -1) quotearg.c:# define SIZE_MAX ((size_t) -1) striconv.c:# define SIZE_MAX ((size_t) -1) Note that stdint.h may not be sufficient to get SIZE_MAX, quoting size_max.h: /* Get SIZE_MAX declaration on systems like Solaris 7/8/9. */ # include limits.h /* Get SIZE_MAX declaration on systems like glibc 2. */ # if HAVE_STDINT_H # include stdint.h # endif /* On systems where these include files don't define it, SIZE_MAX is defined in config.h. */ However given that SIZE_MAX should be in stdint.h according to POSIX, maybe it makes more sense to make sure gnulib's stdint.h replacement is enabled when SIZE_MAX is not provided by the system's stdint.h? And then deprecate size_max.h in favor of stdint. While these already include stdint.h so should probably not redefine fts.c:# define SIZE_MAX ((size_t) -1) getdelim.c:# define SSIZE_MAX ((ssize_t) (SIZE_MAX / 2)) getndelim2.c:# define SSIZE_MAX ((ssize_t) (SIZE_MAX / 2)) SSIZE_MAX should be provided by limit.h, see: http://www.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html The stdint.h documentation doesn't mention SSIZE_MAX: http://www.opengroup.org/onlinepubs/009695399/basedefs/stdint.h.html /Simon
Re: inetutils ChangeLog doc/Makefile.am doc/inetuti...
Alfred M. Szmidt a...@gnu.org writes: doc/fdl.texi is removed below If I'm understanding correctly, removing fdl.texi seems wrong to me. I'm supposing it's created dynamically from a copy in gnulib or somewhere now? But the license can't be updated merely by changing that file. The @copying block has to be updated also. In fact, the @copying block now says 1.2, but (I'm guessing) fdl.texi v1.3 is what gets pulled in. I think the right outcome is: 1) change 1.2 to 1.3 in @copying in inetutils.texi. 2) keep a copy of fdl[-1.3].texi in the repo. 3) in the event that the fdl is updated, both things need to be updated. I don't know of any plausible way to automate it, and updates are so infrequent, it doesn't seem worth the effort. You raise good points and thank you for catching them, I am not sure what we should do. coreutils for example doesn't include fdl.texi, and coreutils is generally our guideline when it comes to these things. Jim and co, what do you think? After the patch I installed to inetutils [1], I think actually the only problem is that the gnulib 'fdl' module is a moving target. That doesn't really work, as Karl explained, since the main manual needs to be updated manually whenever there is a FDL version update in gnulib. So in gnulib, I propose we deprecated 'fdl' and ask maintainers to depend directly on 'fdl-1.3' or whatever version they need. Thoughts? I cc yet another list, bug-gnulib, to get this archived for the gnulib context as well, in case we end up modifying gnulib. Note that gnulib does not contain a 'gpl' or 'lgpl' module, only 'gpl-2.0', 'gpl-3.0', and 'lgpl-2.1'. (Although no lgpl-3.0..) So it seems the 'fdl' module is sub-optimal. /Simon [1] http://lists.gnu.org/archive/html/commit-inetutils/2009-05/msg1.html ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: use maintainer-makefile
Jim Meyering j...@meyering.net writes: Simon Josefsson wrote: Hi! The patch below makes coreutils use maint.mk from gnulib. The maint.mk files are now almost the same, so I hope this is acceptable. You'll have to update the gnulib subproject to 7f62c240e to make bootstrap import the new maint.mk though, and I don't know how that is handled in coreutils. What do you think? Hi Simon, Thank you. Great! To update the submodule to latest, I did this: ( cd gnulib git pull ) then ran git commit --amend -a to add that change to your change set. I didn't know this command, thanks for explaining. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
use maintainer-makefile
Hi! The patch below makes coreutils use maint.mk from gnulib. The
maint.mk files are now almost the same, so I hope this is acceptable.
You'll have to update the gnulib subproject to 7f62c240e to make
bootstrap import the new maint.mk though, and I don't know how that is
handled in coreutils.
What do you think?
/Simon
From 3bb075a79206be398df5ebd28c6d3e708e4bac15 Mon Sep 17 00:00:00 2001
From: Simon Josefsson si...@josefsson.org
Date: Fri, 24 Apr 2009 01:07:43 +0200
Subject: [PATCH] build: use maint.mk from gnulib.
Signed-off-by: Simon Josefsson si...@josefsson.org
---
.gitignore |1 +
bootstrap.conf |1 +
maint.mk | 852
3 files changed, 2 insertions(+), 852 deletions(-)
delete mode 100644 maint.mk
diff --git a/.gitignore b/.gitignore
index 2db10a3..5618f72 100644
--- a/.gitignore
+++ b/.gitignore
@@ -56,6 +56,7 @@ lib/selinux
lib/uniwidth
m4/.cvsignore
m4/.gitignore
+maint.mk
po/*.gmo
po/*.po
po/.gitignore
diff --git a/bootstrap.conf b/bootstrap.conf
index 8475892..475dad4 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -134,6 +134,7 @@ gnulib_modules=
link-follow
long-options
lstat
+ maintainer-makefile
malloc
manywarnings
mbrtowc
diff --git a/maint.mk b/maint.mk
deleted file mode 100644
index 5ef1fe6..000
--- a/maint.mk
+++ /dev/null
@@ -1,852 +0,0 @@
-# -*-Makefile-*-
-# This Makefile fragment tries to be general-purpose enough to be
-# used by at least coreutils, idutils, CPPI, Bison, and Autoconf.
-
-## Copyright (C) 2001-2009 Free Software Foundation, Inc.
-##
-## This program is free software: you can redistribute it and/or modify
-## it under the terms of the GNU General Public License as published by
-## the Free Software Foundation, either version 3 of the License, or
-## (at your option) any later version.
-##
-## This program is distributed in the hope that it will be useful,
-## but WITHOUT ANY WARRANTY; without even the implied warranty of
-## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-## GNU General Public License for more details.
-##
-## You should have received a copy of the GNU General Public License
-## along with this program. If not, see http://www.gnu.org/licenses/.
-
-# This is reported not to work with make-3.79.1
-# ME := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
-ME := maint.mk
-
-# Do not save the original name or timestamp in the .tar.gz file.
-# Use --rsyncable if available.
-gzip_rsyncable := \
- $(shell gzip --help 2/dev/null|grep rsyncable /dev/null echo
--rsyncable)
-GZIP_ENV = '--no-name --best $(gzip_rsyncable)'
-
-GIT = git
-VC = $(GIT)
-VC-tag = git tag -s -m '$(VERSION)'
-
-VC_LIST = $(srcdir)/build-aux/vc-list-files
-
-VC_LIST_EXCEPT = \
- $(VC_LIST) | if test -f .x-$@; then grep -vEf .x-$@; else grep -v ChangeLog;
fi
-
-ifeq ($(origin prev_version_file), undefined)
- prev_version_file = $(srcdir)/.prev-version
-endif
-
-PREV_VERSION := $(shell cat $(prev_version_file))
-VERSION_REGEXP = $(subst .,\.,$(VERSION))
-PREV_VERSION_REGEXP = $(subst .,\.,$(PREV_VERSION))
-
-ifeq ($(VC),$(GIT))
-this-vc-tag = v$(VERSION)
-this-vc-tag-regexp = v$(VERSION_REGEXP)
-else
-tag-package = $(shell echo $(PACKAGE) | tr '[:lower:]' '[:upper:]')
-tag-this-version = $(subst .,_,$(VERSION))
-this-vc-tag = $(tag-package)-$(tag-this-version)
-this-vc-tag-regexp = $(this-vc-tag)
-endif
-my_distdir = $(PACKAGE)-$(VERSION)
-
-# Old releases are stored here.
-release_archive_dir ?= ../release
-
-# Prevent programs like 'sort' from considering distinct strings to be equal.
-# Doing it here saves us from having to set LC_ALL elsewhere in this file.
-export LC_ALL = C
-
-## --- ##
-## Sanity checks. ##
-## --- ##
-
-# Collect the names of rules starting with `sc_'.
-syntax-check-rules := $(shell sed -n 's/^\(sc_[a-zA-Z0-9_-]*\):.*/\1/p' \
-$(srcdir)/$(ME) $(srcdir)/cfg.mk)
-.PHONY: $(syntax-check-rules)
-
-local-checks-available = \
- $(syntax-check-rules) \
- makefile-check
-.PHONY: $(local-checks-available)
-
-# Arrange to print the name of each syntax-checking rule just before running
it.
-$(syntax-check-rules): %: %.m
-$(patsubst %, %.m, $(syntax-check-rules)):
- @echo $(patsubst sc_%.m, %, $@)
-
-local-check := $(filter-out $(local-checks-to-skip), $(local-checks-available))
-
-syntax-check: $(local-check)
-# @grep -nE '# *include (limits|std(def|arg|bool))\.h' \
-# $$(find -type f -name '*.[chly]') \
-#{ echo '$(ME): found conditional include' 12; \
-# exit 1; } || :
-
-# grep -nE '^# *include (string|stdlib)\.h'\
-# $(srcdir)/{lib,src}/*.[chy] \
-#{ echo '$(ME): FIXME' 12; \
-# exit 1; } || :
-# FIXME: don't allow `#include .strings\.h' anywhere
-
-# By default, _prohibit_regexp does
Re: interix 3.5 select bug
Jim Meyering j...@meyering.net writes: + * lib/nanosleep.c (my_usleep): Use 1, not 0, as the first argument. + This avoids a failure on Interix 3.5. Details in + http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/16077 Is this the best solution? It seems that another solution would be to detect this problem, and to replace select if the system's select doesn't work. Is there anything in POSIX whether this should work or not? /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Issue with ls -v / sort -V and strverscmp() usage
Ondřej Vašík [EMAIL PROTECTED] writes: Bruno Haible wrote: Ondřej Vašík wrote: 3) to use/create different function for handling version sort (like rpmvercmp in rpm/rpmlib.h recommended in glibc strverscmp() bugzilla) Does rpmvercmp have the above property? Similar. It separates name, epoch, version, release and architecture and compares those. Uses subsections of alphanum segments separated by non-alphanum chars. When same segment have different types, numeric is always considered as newer than alpha or alphanum. This solves issue with suffix, but changes behaviour for prefix (name) - as fully numeric project/file name will be considered as newer than alphanum project/file name and will be listed after alphanumeric names. Do you know if the debian dpkg --compare-versions sort-order is different from the rpm sort-order? In any case, adding a new gnulib module with the documented semantic 'sort filenames in an order that makes sense when comparing software versions' seems appropriate. That allows us to change the logic later if we find examples that sorts incorrectly. Possibly the function could take a parameter to sort in RPM-mode or in DPKG-mode, if those algorithms are substantially different, and both outputs are needed by different applications. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: sort --version-sort
Bruce Korb [EMAIL PROTECTED] writes: Here is the usage text from my latest --sort=version patch (attached), now that paperwork is in order. What is the status of this work? I have wanted to use sort this way several times but became disappointed that there is no support for it every time. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
base64: fix typos in comment
Hi! I looked over the base64 tool a bit, and found two minor issues. Thanks, /Simon From 5621aab03b8187c50cab094369e0faaf7e01d14c Mon Sep 17 00:00:00 2001 From: Simon Josefsson [EMAIL PROTECTED] Date: Sun, 1 Jun 2008 13:08:20 +0200 Subject: [PATCH] Improve comments. --- src/base64.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/base64.c b/src/base64.c index c9e50cf..a3a7345 100644 --- a/src/base64.c +++ b/src/base64.c @@ -240,9 +240,9 @@ main (int argc, char **argv) FILE *input_fh; const char *infile; - /* True if --decode has bene given and we should decode data. */ + /* True if --decode has been given and we should decode data. */ bool decode = false; - /* True if we should ignore non-alphabetic characters. */ + /* True if we should ignore non-base64-alphabetic characters. */ bool ignore_garbage = false; /* Wrap encoded base64 data around the 76:th column, by default. */ uintmax_t wrap_column = 76; -- 1.5.5.1 ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Feature request - base64 Filename Safe Alphabet
Christopher Kerr [EMAIL PROTECTED] writes: After being burned by using `head -c6 /dev/urandom | base64` as part of a directory name, I realised that it would be useful if base64 had an option to generate URL and Filename safe encodings, as specified in RFC 3548 section 4. This would make cat FILE | base64 --filename-safe equivalent to cat FILE | base64 | tr '+/' '-_' using the current coreutils tools. I think --filename-safe is a good idea. The documentation should discuss the potential for generating files starting with '-' or '--'. Patching gnulib's base64.c to support an arbitrary alphabet seems messy. Patches welcome though. Regarding the discussion of different characters to use, let me add that '+' is not a URI safe character, so it would be unsafe from that aspect. I believe the parameter name clash is the least problematic consequence that we can chose. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64: fix typos in comment
Jim Meyering [EMAIL PROTECTED] writes:
Simon Josefsson [EMAIL PROTECTED] wrote:
Hi! I looked over the base64 tool a bit, and found two minor issues.
Subject: [PATCH] Improve comments.
...
Thanks! Applied.
That was quick, thanks. I went over the documentation as well.
/Simon
From 450da29a8766003e35891e7f5209e4fa135d2845 Mon Sep 17 00:00:00 2001
From: Simon Josefsson [EMAIL PROTECTED]
Date: Sun, 1 Jun 2008 13:51:14 +0200
Subject: [PATCH] use RFC 4648 as base64 specification, and move the URL down a
bit in the text
---
doc/coreutils.texi |7 ---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index dbc8a8b..020cb00 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -1860,8 +1860,7 @@ address.
@command{base64} transforms data read from a file, or standard input,
into (or from) base64 encoded form. The base64 encoded form uses
-printable @acronym{ASCII} characters to represent binary data, see
[EMAIL PROTECTED]://ftp.rfc-editor.org/in-notes/rfc3548.txt, RFC 3548}.
+printable @acronym{ASCII} characters to represent binary data.
Synopses:
@smallexample
@@ -1869,7 +1868,9 @@ base64 [EMAIL PROTECTED]@dots{} [EMAIL PROTECTED]
base64 --decode [EMAIL PROTECTED]@dots{} [EMAIL PROTECTED]
@end smallexample
-The base64 encoding expands data to roughly 133% of the original.
+The base64 encoding expands data to roughly 133% of the original. The
+format conforms to
[EMAIL PROTECTED]://ftp.rfc-editor.org/in-notes/rfc4648.txt, RFC 4648}.
The program accepts the following options. Also see @ref{Common options}.
--
1.5.5.1
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Feature request - base64 Filename Safe Alphabet
Bo Borgerson [EMAIL PROTECTED] writes: Jim Meyering wrote: I found strict_newlines to be a little unclear. If you use something like ignore_newlines instead, that's not only clearer to me, but with its reversed semantics it also lets you avoid three negations. Thanks, that's much nicer. The attached patch contains this change and is rebased against the current HEAD. I've also made this available via: $ git fetch git://repo.or.cz/coreutils/bo.git base64-merge:base64-merge Hi Bo. Many thanks for picking up the slack here, I should have improved gnulib's base64 more with coreutils long time ago. Your patch is rather difficult to read for me, since I'm not that familiar with the coreutils changes, and more importantly: to be applied to gnulib, I need a patch against gnulib. Would you mind creating a patchset that applies to the gnulib git repository? If it looks fine to me, I'll apply it and then coreutils can sync against it without any local modifications. I suspect your patch do things the way I suggested in the post to the gnulib list some time ago, which is nice. Thanks, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH] Generalize GNUmakefile, ...
Eric Blake [EMAIL PROTECTED] writes: [adding gnulib] According to Jim Meyering on 3/12/2008 5:39 AM: | Hi, Hi Jim, Simon, others, | | I'm planing to make the following change in coreutils | so that autoconf can then use the exact same GNUmakefile. Hmm. I was about to try to add the latest coreutils/autoconf/m4 GNUmakefile to gnulib, to make it easier to share. But gnulib already provides build-aux/GNUmakefile as part of the maintainer-makefile module, added by Simon a while ago (and originally borrowed from coreutils). Hi! Thanks for looking into this, and sorry for slow response. Merging them would be useful. There are some things in coreutils that I'd like to have, but haven't had time to merge into gnulib. Differences between the two: the gnulib version uses maint.mk rather than Makefile.maint as its file of maintainer-specific rules, and includes maint.mk as part of the module The reason was that my emacs appear to treat *.mk as makefiles, but didn't with *.maint. I don't care strongly, but do prefer maint.mk. the gnulib version makes the existence of maint-cfg.mk (cf. Makefile.cfg) option, rather than required I don't care strongly about this. The name 'maint-cfg.mk' haven't been the best though, since it clashes with name completion of main.mk. Is cfg.mk too generic? I think this would be a good opportunity to change the name. the coreutils version uses build-aux/git-version-gen to form nice intra-release version numbering, while minimizing when a full autoreconf must be performed to pick up the latest version number This is one thing I'd like to have in my packages as well. the gnulib version uses '.DEFAULT_GOAL:=' rather than 'all:' when Makefile is not present to trigger the error message about needing to run configure in all cases, rather than just a few I don't care strongly, but this approach allows maint-cfg.mk to override the default goal. I use this in all of my projects to invoke autoreconf etc if they haven't been invoked yet. (I don't use the bootstrap approach since, as far as I can tell, it makes it difficult to prepare security releases that are bit-by-bit identical with the earlier release, except for the patch to fix the security problem.) the coreutils version conditionally uses AC_CONFIG_LINKS in configure.ac, along with EXTRA_DIST and distclean-local in Makefile.am, to support GNUmakefile usage even in VPATH builds I can't comment on that, but if it doesn't cause problems for me, I'm for it. :) Is it worth trying to re-merge these two approaches, since they forked from a common ancestor? Yes please. At this point, I'm thinking of having two modules, one that distributes just GNUmakefile, and leaving maintainer-makefile to depend on the new module and just provide maint.mk. The GNUmakefile seems rather small, but if you think separating the modules gives an advantage, I don't object. There needs to be some hooks in GNUmakefile to load the maintainer-makefile though, I think. Bruno Haible [EMAIL PROTECTED] writes: But since a GNUmakefile in the top directory of gnulib would hide the Makefile of gnulib, I propose to move it from build-aux/GNUmakefile to top/GNUmakefile, and enhance gnulib-tool with a filename rewriting rule that transforms $GNULIBDIR/top/GNUmakefile -- $PACKAGEDIR/GNUmakefile I think this is a good idea. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Errors in the coreutils-6.9 PO file
Eric Blake [EMAIL PROTECTED] writes: According to Clytie Siddall on 10/24/2007 2:37 AM: 2. #: lib/gai_strerror.c:52 msgid Servname not supported for ai_socktype Servname is ambiguous: Service name or Server name? You may well get inaccurate translations. Simon? This is your gnulib module. The string comes from glibc (sysdeps/posix/gai_strerror-strs.h). Maybe it could be solved upstream? I'm not sure what it should say, I think it should be Service name. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Sync maintainer-makefile files (was: Re: maint.mk v. gzip 1.2.4)
Simon Josefsson [EMAIL PROTECTED] writes:
If we make maint.mk minimal, it would be excellent to make coreutils'
Makefile.maint do 'include maint.mk'. This was the idea all along.
I'll suggest patches for gnulib/coreutils to achieve this.
Below are two set of patches. The first one is against coreutils, to
make it start use gnulib's maint.mk file. The code that is removed from
coreutils (gzip + lc_all) is in gnulib's build-aux/maint.mk.
The second set of patches is against gnulib to remove most of the rules
in coreutils, to avoid duplication. This removes some features from
gnulib's maint.mk, but has anyone been using them heavily? If so, we
can add them again by making small incremental patches against coreutils
and gnulib.
Thoughts?
/Simon
Patch against coreutils
diff --git a/Makefile.maint b/Makefile.maint
index ba39dec..0600efb 100644
--- a/Makefile.maint
+++ b/Makefile.maint
@@ -17,16 +17,12 @@
## You should have received a copy of the GNU General Public License
## along with this program. If not, see http://www.gnu.org/licenses/.
+include $(srcdir)/build-aux/maint.mk
+
# This is reported not to work with make-3.79.1
# ME := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
ME := Makefile.maint
-# Do not save the original name or timestamp in the .tar.gz file.
-# Use --rsyncable if available.
-gzip_rsyncable := \
- $(shell gzip --help 2/dev/null|grep rsyncable /dev/null echo
--rsyncable)
-GZIP_ENV = '--no-name --best $(gzip_rsyncable)'
-
CVS = cvs
GIT = git
VC = $(GIT)
@@ -59,10 +55,6 @@ my_distdir = $(PACKAGE)-$(VERSION)
# Used for diffs and xdeltas.
release_archive_dir ?= ../release
-# Prevent programs like 'sort' from considering distinct strings to be equal.
-# Doing it here saves us from having to set LC_ALL elsewhere in this file.
-export LC_ALL = C
-
## --- ##
diff --git a/bootstrap.conf b/bootstrap.conf
index d07c23e..9f63008 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -58,6 +58,7 @@ gnulib_modules=
inttostr inttypes isapipe
lchmod lchown lib-ignore linebuffer link-follow
long-options lstat malloc mbswidth memcasecmp mempcpy
+ maintainer-makefile
memrchr mgetgroups
mkancesdirs mkdir mkdir-p mkstemp mktime modechange
mountlist mpsort obstack pathmax perl physmem posixtm posixver putenv
diff --git a/build-aux/.gitignore b/build-aux/.gitignore
index fc92044..1629629 100644
--- a/build-aux/.gitignore
+++ b/build-aux/.gitignore
@@ -1,3 +1,4 @@
+GNUmakefile
announce-gen
compile
config.guess
@@ -7,6 +8,7 @@ depcomp
gnupload
install-sh
link-warning.h
+maint.mk
mdate-sh
missing
texinfo.tex
Patch against gnulib:
diff --git a/build-aux/maint.mk b/build-aux/maint.mk
index 662b371..45ea3ad 100644
--- a/build-aux/maint.mk
+++ b/build-aux/maint.mk
@@ -19,14 +19,6 @@
ME := maint.mk
-# List of all C-like source code files that will be tested for
-# stylistic errors. You may want to define this to something
-# more complex in Makefile.cfg.
-C_SOURCES ?= $(shell find . -name '*.[chly]')
-
-# Add some more files to check, typically set in Makefile.cfg.
-C_SOURCES += $(C_SOURCES_ADD)
-
# Do not save the original name or timestamp in the .tar.gz file.
# Use --rsyncable if available.
gzip_rsyncable := \
@@ -37,74 +29,6 @@ GZIP_ENV = '--no-name --best $(gzip_rsyncable)'
# Doing it here saves us from having to set LC_ALL elsewhere in this file.
export LC_ALL = C
-# Casting arguments to free is never necessary.
-sc_cast_of_argument_to_free:
- @grep -nE '\free *\( *\(' $(C_SOURCES) \
- { echo '$(ME): don'\''t cast free argument' 12; \
- exit 1; } || :
-
-sc_cast_of_x_alloc_return_value:
- @grep -nE '\*\) *x(m|c|re)alloc\' $(C_SOURCES) \
- { echo '$(ME): don'\''t cast x*alloc return value' 12; \
- exit 1; } || :
-
-sc_cast_of_alloca_return_value:
- @grep -nE '\*\) *alloca\' $(C_SOURCES) \
- { echo '$(ME): don'\''t cast alloca return value' 12; \
- exit 1; } || :
-
-sc_space_tab:
- @grep -n '[ ] ' $(C_SOURCES)\
- { echo '$(ME): found SPACE-TAB sequence; remove the SPACE'\
- 12; exit 1; } || :
-
-# Don't use the old ato* functions in `real' code.
-# They provide no error checking mechanism.
-# Instead, use strto* functions.
-sc_prohibit_atoi_atof:
- @grep -nE '\ato([filq]|ll)\' $(C_SOURCES) \
- { echo '$(ME): do not use ato''f, ato''i, ato''l, ato''ll, or ato''q'
\
- 12; exit 1; } || :
-
-# Using EXIT_SUCCESS as the first argument to error is misleading,
-# since when that parameter is 0, error does not exit. Use `0' instead.
-sc_error_exit_success:
- @grep -nF 'error (EXIT_SUCCESS,' $(C_SOURCES) \
- { echo '$(ME): found error (EXIT_SUCCESS' 12; \
- exit 1
Re: gnulib getcwd patch needed for Solaris 10 getcwd
Matthew Woehlke [EMAIL PROTECTED] writes: Jim Meyering wrote: Bad timing. I was about to make a test release. If anyone else has access to a Solaris 10 system (x86 or otherwise), with Sun C, would you please try the latest snapshot? http://meyering.net/cu/ FYI if you feel the need for more test results, I can do builds on both SPARC and x86 for you with the native compilers, on both NFS and UFS, but not until next week. We do need more autobuilders of gnulib, though, so if you can handle multi-hour builds every day, consider running a script similar to the one described at: http://autobuild.josefsson.org/ /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [Bug 204567] New: base64 -d doesn't accept base64 output
I agree that this is a bug, and I'm interested in fixing this, but I have little free time right now. I'd prefer that --wrap can be used to indicate at what position the tool should expect CR/LF on decoding. If there isn't a CR/LF at that position, it should abort. Also, there should be one new parameter to make the tool accept one additional CR/LF after the data when decoding, and it should probably default to on, or possibly only default to on when --wrap is used. If anyone wants to take a stab at implementing this, that would be great... I'll get to it eventually, otherwise. /Simon Jim Meyering [EMAIL PROTECTED] writes: In https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=204567, James Antill wrote: Summary: base64 -d doesn't accept base64 output Description of problem: base64 -d doesn't accept returns (which the base64 encoding should put every 76 characters and at the end of the encoding). Version-Release number of selected component (if applicable): coreutils-5.97-6 How reproducible: Always Steps to Reproduce: 1. echo x | base64 | base64 -d 2. perl -le 'print a x 80' | base64 | base64 -d Additional info: base64 -id does work but (IMNSHO) the output of base64 and what the rfc says you should do should hardly be clasified as garbage. However this does present the obvious fix of looking for '\n' (and skipping it) on the else clause of the ignore_garbage check. Thanks for the report and the patch. I suspect you know it already, but for the record, an alternative is to filter out newlines before running base64 -d or to encode with --wrap=0, to avoid emitting newlines altogether. But your point is valid. --- Additional Comments From [EMAIL PROTECTED] Created an attachment (id=135184) -- (https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=135184action=view) Simple fix for base64 decoding return's This is a simple fix, although not very efficient and allows some things through that shouldn't be allowed without -i. On the other hand it's so simple it's obviously OK to apply. I agree that implying base64-added newlines are garbage is not aesthetically pleasing :) I'd prefer a change that adds an option to ignore newlines (or maybe an arbitrary set of bytes) efficiently. Of course, that would require more invasive changes. Simon Josefsson (Cc'd) wrote both the program and much of the base64 gnulib module, so maybe he'll be interested in looking at this. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: coreutils-6.0 on BeOS (6)
Bruno Haible [EMAIL PROTECTED] writes: Simon Josefsson wrote: The reason is that BeOS does not have PF_INET, only AF_INET, but usually they have the same values. Also it doesn't have PF_UNSPEC. Does it AF_UNSPEC? Did you grep the entire /usr/include tree to find PF_INET or PF_UNSPEC? Maybe they are in some non-standard header. It has neither AF_UNSPEC nor PF_UNSPEC, in no header. Ok, thanks. + #ifdef PF_UNSPEC /* BeOS lacks PF_UNSPEC. */ if (family == PF_UNSPEC) return true; + #endif I'm not sure this will do the right thing. Usually getaddrinfo is called with hints structure that is zeroed out, and only the relevant flags asserted. If family isn't asserted, it usually means take any family. I see. Then what about this patch? It compiles fine on BeOS. Looks good to me. Please install it. /Simon 2006-08-26 Bruno Haible [EMAIL PROTECTED] Simon Josefsson [EMAIL PROTECTED] * getaddrinfo.c (PF_INET, PF_UNSPEC): New macros. *** coreutils-6.2-cvs/lib/getaddrinfo.c 2006-08-26 21:52:06.0 +0200 --- coreutils-6.2-beos/lib/getaddrinfo.c 2006-08-26 23:35:16.0 +0200 *** *** 42,47 --- 42,56 #include snprintf.h #include strdup.h + /* BeOS has AF_INET, but not PF_INET. */ + #ifndef PF_INET + # define PF_INET AF_INET + #endif + /* BeOS also lacks PF_UNSPEC. */ + #ifndef PF_UNSPEC + # define PF_UNSPEC 0 + #endif + #if defined _WIN32 || defined __WIN32__ # define WIN32_NATIVE #endif ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: coreutils-6.0 on BeOS (6)
Bruno Haible [EMAIL PROTECTED] writes: The reason is that BeOS does not have PF_INET, only AF_INET, but usually they have the same values. Also it doesn't have PF_UNSPEC. Does it AF_UNSPEC? Did you grep the entire /usr/include tree to find PF_INET or PF_UNSPEC? Maybe they are in some non-standard header. This is quite a hacky fix, but I don't know what value could be used for PF_UNSPEC if we were to make a sys/socket.h override. Perhaps 0. + /* BeOS has AF_INET, but not PF_INET. */ + #ifndef PF_INET + # define PF_INET AF_INET + #endif Seems reasonable. + #ifdef PF_UNSPEC /* BeOS lacks PF_UNSPEC. */ if (family == PF_UNSPEC) return true; + #endif I'm not sure this will do the right thing. Usually getaddrinfo is called with hints structure that is zeroed out, and only the relevant flags asserted. If family isn't asserted, it usually means take any family. With the above change, I don't think getaddrinfo will work at all on BeOS. It will just return EAI_FAMILY. Most callers (rightly) doesn't specify a family. The best would be if AF_UNSPEC existed, then we can map PF_UNSPEC to it. If not, I think we'd might as well bind it to 0 directly. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: gnupload
If coreutils is updated to sync gnupload from gnulib, which I think would be a good idea, please drop gnupload from gnulib's config/srclist.txt. /Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Support for SIGSTOP/SIGCONT in nice?
This is a feature request for some non-essential functionality in 'nice'. I understand if you think this is too uninteresting and/or just add unnecessary code complexity. Your thoughts on the idea are appreciated in any case... Frequently when I want to run a long-running I/O-bound job (i.e., rsync of the entire disk to a second disk) on a busy machine (i.e., mail and web server), the load increases to ~10 from a normal 1. Using 'nice' doesn't help. I don't know why this is, but I suspect the kernel scheduler is sub-optimal for disk-bound jobs. Would it be useful to consider extending nice so it can send SIGSTOP and SIGCONT signals in configurable intervals? Having rsync run for 30 seconds, sleep for 15, run for 30 seconds, etc, seem to avoid slowing down the system to unusable levels. I don't know what the syntax would be, but an 'advanced' interface might be: nice --signal-length 45s --signal-pattern rrs foo Which would mean to have 'foo' run for 30 seconds, sleep for 15 seconds, etc. Comments? Thanks. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Support for SIGSTOP/SIGCONT in nice?
Jim Meyering [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] wrote: ... Frequently when I want to run a long-running I/O-bound job (i.e., rsync of the entire disk to a second disk) on a busy machine (i.e., mail and web server), the load increases to ~10 from a normal 1. Using 'nice' doesn't help. I don't know why this is, but I suspect the kernel scheduler is sub-optimal for disk-bound jobs. Would it be useful to consider extending nice so it can send SIGSTOP and SIGCONT signals in configurable intervals? Having rsync run for 30 seconds, sleep for 15, run for 30 seconds, etc, seem to avoid slowing down the system to unusable levels. Hi Simon, Have you already tried using rsync's existing option to limit bandwidth? rsync --bwlimit=500 ... Hi Jim! I had forgotten about it. It seem to have solved the actual problem I had. Still, maybe not all programs have similar rate-limiting features, so that this nice-extension may be useful anyway... Thanks, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Support for SIGSTOP/SIGCONT in nice?
Andreas Schwab [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] writes: Frequently when I want to run a long-running I/O-bound job (i.e., rsync of the entire disk to a second disk) on a busy machine (i.e., mail and web server), the load increases to ~10 from a normal 1. Assuming you are running on Linux, this is because there are about 10 processes waiting for disk I/O. Using 'nice' doesn't help. I don't know why this is, but I suspect the kernel scheduler is sub-optimal for disk-bound jobs. You probably want to change the I/O scheduling class for the process. This can be done with ionice (which is Linux specific, included in the util-linux package at least on SL 10.1). Thanks for the hint. I couldn't find it in Debian, but some googling revealed: http://www.kernel.org/pub/linux/utils/util-linux/testing/ ionice is part of the latest pre-releases. Unfortunately, my kernel on the system didn't support that syscall so it didn't work. Regards, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Support for SIGSTOP/SIGCONT in nice?
[EMAIL PROTECTED] (Bob Proulx) writes: Jim Meyering wrote: Simon Josefsson wrote: Frequently when I want to run a long-running I/O-bound job (i.e., rsync of the entire disk to a second disk) on a busy machine (i.e., mail and web server), the load increases to ~10 from a normal 1. Using 'nice' doesn't help. I don't know why this is, but I suspect the kernel scheduler is sub-optimal for disk-bound jobs. Would it be useful to consider extending nice so it can send SIGSTOP and SIGCONT signals in configurable intervals? Having rsync run for 30 seconds, sleep for 15, run for 30 seconds, etc, seem to avoid slowing down the system to unusable levels. Have you already tried using rsync's existing option to limit bandwidth? rsync --bwlimit=500 ... While doing some research into this problem I found that this issue has been reported to rsync before. Here is a previous thread discussing it. http://lists.samba.org/archive/rsync/2004-February/008638.html In the thread is an interesting perl snippet specifically for this purpose. It monitors load and stops and starts rsync as needed. http://lists.samba.org/archive/rsync/2004-February/008649.html I would be inclined to see if the poster involved there made any additional progress into this issue with rsync. It has been two years and so perhaps they found a good way to deal with this problem already. Wow, just the same script I was about to write... However, --bwlimit do seem to have solved this problem for me. ionice (with the kernel scheduler changes to go with it) seem to be the more generic solution, at least for Linux. I'm not sure there is a non-Linux-specific generic solution. Perhaps nice in Coreutils could be enhanced to support the ionice syscall on Linux (although having separate commands might be cleaner). I was sure I have seen a generic script program that would cycle another program on and off with sigstop/sigcont just as you are asking for as a programmable load controller. But it slips my mind at the moment. I did not find the one I was remembering. [Any else have any suggestions?] But I did find something that seemed interesting so I will include a pointer to it. http://www.pattosoft.com.au/jason/idleize/ That seems useful... But it probably works better with CPU-bound stuff than with IO-bound stuff. Thanks. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Jim Meyering [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] wrote: Jim Meyering [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] wrote: Here are updated tests. They test several corner cases, but still not comprehensive. Thank you! Applied. With the patch below, it builds and passes self-tests fine here. Thanks. How about this instead, in case someone wants to use a wrap width that fits in uintmax_t, but not in an unsigned long int? That seem better to me, so please install it. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Jim Meyering [EMAIL PROTECTED] writes:
Simon Josefsson [EMAIL PROTECTED] wrote:
Here is an updated patch for Coreutils to provide the base64 command.
I believe I have addressed all major comments given earlier, I'm
Cc:ing all people who helped me, maybe they can help review it again.
The lib/base64.? files are from gnulib.
Warning: it has not been tested much at all!
Thank you!
I've made some minor changes e.g., to get past coreutils `make distcheck':
* src/base64.c: Don't include .h files already included by system.h:
string.h, stdlib.h, stdbool.h, limits.h, errno.h.
Include system.h before the other lib/*.h header files.
Include sys/types.h before system.h.
(wrap_write): Remove declaration of unused local, initial_column.
(wrap_write): Correct declaration syntax: s/size_t * V/size_t *V/.
* README: Add base64 to the list.
Excellent!
Plus, I indented a few cpp directives in lib/base64.h,
to make them consistent with the first #define.
I feeded gnulib with these.
Would you please write a test script to go along with that?
I'd prefer that you model the new file, say tests/misc/base64,
after one of the existing ones that uses the Coreutils.pm module.
For example, tests/misc/expand is nice and small.
I forgot to include it, it was part of the first patch. Is the below
sufficient?
Thanks!
Index: tests/base64/Makefile.am
===
RCS file: tests/base64/Makefile.am
diff -N tests/base64/Makefile.am
--- /dev/null 1 Jan 1970 00:00:00 -
+++ tests/base64/Makefile.am27 Feb 2006 11:30:12 -
@@ -0,0 +1,12 @@
+## Process this file with automake to produce Makefile.in -*-Makefile-*-.
+
+EXTRA_DIST = $(TESTS)
+
+TESTS_ENVIRONMENT = \
+ top_srcdir=$(top_srcdir) \
+ srcdir=$(srcdir) \
+ PERL=$(PERL) \
+ PATH=`pwd`/../../src$(PATH_SEPARATOR)$$PATH \
+ PROG=base64
+
+TESTS = basic-1
Index: tests/base64/basic-1
===
RCS file: tests/base64/basic-1
diff -N tests/base64/basic-1
--- /dev/null 1 Jan 1970 00:00:00 -
+++ tests/base64/basic-127 Feb 2006 11:30:12 -
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+: ${PERL=perl}
+: ${srcdir=.}
+
+$PERL -e 1 /dev/null 21 || {
+ echo 12 $0: configure didn't find a usable version of Perl, \
+so can't run this test
+ exit 77
+}
+
+exec $PERL -w -I$srcdir/.. -MCoreutils -- - \EOF
+require 5.003;
+use strict;
+
+(my $program_name = $0) =~ s|.*/||;
+
+# Turn off localisation of executable's ouput.
[EMAIL PROTECTED](LANGUAGE LANG LC_ALL)} = ('C') x 3;
+
+my $degenerate = d41d8cd98f00b204e9800998ecf8427e;
+
+my @Tests =
+(
+ ['empty', {IN= {f= ''}},{OUT=}],
+ ['2', {IN= {f= 'a'}}, {OUT=YQ==\n}],
+);
+
+my $save_temps = $ENV{DEBUG};
+my $verbose = $ENV{VERBOSE};
+
+my $prog = $ENV{PROG} || die $0: \$PROG not specified in environment\n;
+my $fail = run_tests ($program_name, $prog, [EMAIL PROTECTED], $save_temps,
$verbose);
+exit $fail;
+EOF
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Jim Meyering [EMAIL PROTECTED] writes:
Would you please write a test script to go along with that?
I'd prefer that you model the new file, say tests/misc/base64,
after one of the existing ones that uses the Coreutils.pm module.
For example, tests/misc/expand is nice and small.
I forgot to include it, it was part of the first patch. Is the below
sufficient?
It'd be nice if you would add examples demonstrating
use of the options. For example, I tried this:
$ seq 100|./base64 --wr=0
./base64: invalid wrap size: 0
[Exit 1]
A basic test might have caught that.
Here are updated tests. They test several corner cases, but still not
comprehensive.
Index: tests/base64/Makefile.am
===
RCS file: tests/base64/Makefile.am
diff -N tests/base64/Makefile.am
--- /dev/null 1 Jan 1970 00:00:00 -
+++ tests/base64/Makefile.am27 Feb 2006 13:33:05 -
@@ -0,0 +1,12 @@
+## Process this file with automake to produce Makefile.in -*-Makefile-*-.
+
+EXTRA_DIST = $(TESTS)
+
+TESTS_ENVIRONMENT = \
+ top_srcdir=$(top_srcdir) \
+ srcdir=$(srcdir) \
+ PERL=$(PERL) \
+ PATH=`pwd`/../../src$(PATH_SEPARATOR)$$PATH \
+ PROG=base64
+
+TESTS = basic-1
Index: tests/base64/basic-1
===
RCS file: tests/base64/basic-1
diff -N tests/base64/basic-1
--- /dev/null 1 Jan 1970 00:00:00 -
+++ tests/base64/basic-127 Feb 2006 13:33:05 -
@@ -0,0 +1,51 @@
+#!/bin/sh
+# -*- perl -*-
+# Exercise base64.
+
+: ${PERL=perl}
+: ${srcdir=.}
+
+$PERL -e 1 /dev/null 21 || {
+ echo 12 $0: configure didn't find a usable version of Perl, \
+so can't run this test
+ exit 77
+}
+
+exec $PERL -w -I$srcdir/.. -MCoreutils -- - \EOF
+require 5.003;
+use strict;
+
+(my $program_name = $0) =~ s|.*/||;
+
+# Turn off localisation of executable's ouput.
[EMAIL PROTECTED](LANGUAGE LANG LC_ALL)} = ('C') x 3;
+
+my $degenerate = d41d8cd98f00b204e9800998ecf8427e;
+
+my @Tests =
+(
+ ['empty', {IN=''}, {OUT=}],
+ ['inout', {IN='a'}, {OUT=YQ==\n}],
+ ['wrap', '--wrap 0', {IN='foo'}, {OUT='Zm9v'}],
+ ['wrap5-39', '--wrap 5', {IN='a' x 39},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFh\n}],
+ ['wrap5-40', '--wrap 5', {IN='a' x 40},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYQ=\n=\n}],
+ ['wrap5-41', '--wrap 5', {IN='a' x 41},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWE\n=\n}],
+ ['wrap5-42', '--wrap 5', {IN='a' x 42},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nh\n}],
+ ['wrap5-43', '--wrap 5', {IN='a' x 43},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYQ==\n}],
+ ['wrap5-44', '--wrap 5', {IN='a' x 44},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWE=\n}],
+ ['wrap5-45', '--wrap 5', {IN='a' x 45},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\n}],
+ ['wrap5-46', '--wrap 5', {IN='a' x 46},
{OUT=YWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYWFhY\nWFhYW\nFhYWF\nhYWFh\nYQ==\n}],
+ ['baddecode', '--decode', {IN='a'}, {OUT=}, {ERR_SUBST = 's/.*:
invalid input//'}, {ERR = \n}, {EXIT = 1}],
+ ['baddecode2', '--decode', {IN='ab'}, {OUT=i}, {ERR_SUBST = 's/.*:
invalid input//'}, {ERR = \n}, {EXIT = 1}],
+ ['baddecode3', '--decode', {IN='Zzz'}, {OUT=g}, {ERR_SUBST = 's/.*:
invalid input//'}, {ERR = \n}, {EXIT = 1}],
+ ['baddecode4', '--decode', {IN='Zz='}, {OUT=g}, {ERR_SUBST = 's/.*:
invalid input//'}, {ERR = \n}, {EXIT = 1}],
+ ['baddecode5', '--decode', {IN='Z==='}, {OUT=}, {ERR_SUBST = 's/.*:
invalid input//'}, {ERR = \n}, {EXIT = 1}]
+);
+
+my $save_temps = $ENV{DEBUG};
+my $verbose = $ENV{VERBOSE};
+
+my $prog = $ENV{PROG} || die $0: \$PROG not specified in environment\n;
+my $fail = run_tests ($program_name, $prog, [EMAIL PROTECTED], $save_temps,
$verbose);
+exit $fail;
+EOF
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Here is an updated patch for Coreutils to provide the base64 command. I believe I have addressed all major comments given earlier, I'm Cc:ing all people who helped me, maybe they can help review it again. The lib/base64.? files are from gnulib. Warning: it has not been tested much at all! Thanks! /ChangeLog: 2006-02-17 Simon Josefsson [EMAIL PROTECTED] * AUTHORS: Mention base64. * NEWS: Likewise. * man/Makefile.am: Build base64.1. * man/base64.x: New file. * src/Makefile.am (bin_PROGRAMS): Add base64. * src/base64.c: New file. doc/ChangeLog: 2006-02-17 Simon Josefsson [EMAIL PROTECTED] * coreutils.texi: Add base64 section. lib/ChangeLog: 2006-02-17 Simon Josefsson [EMAIL PROTECTED] * base64.h, base64.c: New files. * Makefile.am (libcoreutils_a_SOURCES): Add base64.h and base64.c. Index: AUTHORS === RCS file: /cvsroot/coreutils/coreutils/AUTHORS,v retrieving revision 1.8 diff -u -p -r1.8 AUTHORS --- AUTHORS 23 Oct 2005 15:37:53 - 1.8 +++ AUTHORS 17 Feb 2006 17:45:08 - @@ -1,6 +1,7 @@ Here are the names of the programs in this package, each followed by the name(s) of its author(s). +base64: Simon Josefsson basename: FIXME unknown cat: Torbjorn Granlund, Richard M. Stallman chgrp: David MacKenzie, Jim Meyering Index: NEWS === RCS file: /cvsroot/coreutils/coreutils/NEWS,v retrieving revision 1.359 diff -u -p -r1.359 NEWS --- NEWS11 Feb 2006 19:25:26 - 1.359 +++ NEWS17 Feb 2006 17:45:08 - @@ -64,6 +64,7 @@ GNU coreutils NEWS sha256sum: print or check a SHA256 (256-bit) checksum sha384sum: print or check a SHA384 (384-bit) checksum sha512sum: print or check a SHA512 (512-bit) checksum + base64: base64 encoding and decoding (RFC 3548) functionality. ** New features Index: man/Makefile.am === RCS file: /cvsroot/coreutils/coreutils/man/Makefile.am,v retrieving revision 1.40 diff -u -p -r1.40 Makefile.am --- man/Makefile.am 23 Oct 2005 15:17:41 - 1.40 +++ man/Makefile.am 17 Feb 2006 17:45:08 - @@ -1,6 +1,6 @@ ## Process this file with automake to produce Makefile.in -*-Makefile-*- dist_man_MANS = \ - basename.1 cat.1 chgrp.1 chmod.1 chown.1 chroot.1 cksum.1 comm.1 \ + base64.1 basename.1 cat.1 chgrp.1 chmod.1 chown.1 chroot.1 cksum.1 comm.1 \ cp.1 csplit.1 cut.1 date.1 dd.1 df.1 dir.1 dircolors.1 dirname.1 du.1 \ echo.1 env.1 expand.1 expr.1 factor.1 false.1 fmt.1 fold.1 groups.1 \ head.1 hostid.1 hostname.1 id.1 install.1 join.1 kill.1 \ @@ -21,6 +21,7 @@ MAINTAINERCLEANFILES = $(man_MANS) # Depend on configure.ac to get version number changes. common_dep = $(top_srcdir)/configure.ac +base64.1: $(common_dep) $(srcdir)/base64.x ../src/base64.c basename.1:$(common_dep) $(srcdir)/basename.x../src/basename.c cat.1: $(common_dep) $(srcdir)/cat.x ../src/cat.c chgrp.1: $(common_dep) $(srcdir)/chgrp.x ../src/chgrp.c Index: man/base64.x === RCS file: man/base64.x diff -N man/base64.x --- /dev/null 1 Jan 1970 00:00:00 - +++ man/base64.x17 Feb 2006 17:45:08 - @@ -0,0 +1,4 @@ +[NAME] +base64 \- base64 encode/decode data and print to standard output +[DESCRIPTION] +.\ Add any additional description here Index: src/Makefile.am === RCS file: /cvsroot/coreutils/coreutils/src/Makefile.am,v retrieving revision 1.66 diff -u -p -r1.66 Makefile.am --- src/Makefile.am 11 Feb 2006 20:58:09 - 1.66 +++ src/Makefile.am 17 Feb 2006 17:45:08 - @@ -29,6 +29,7 @@ bin_PROGRAMS = [ chgrp chown chmod cp dd basename date dirname echo env expr factor false \ hostname id kill logname pathchk printenv printf pwd seq sleep tee \ test true tty whoami yes \ + base64 \ $(OPTIONAL_BIN_PROGS) $(DF_PROG) noinst_PROGRAMS = setuidgid Index: src/base64.c === RCS file: src/base64.c diff -N src/base64.c --- /dev/null 1 Jan 1970 00:00:00 - +++ src/base64.c17 Feb 2006 17:45:08 - @@ -0,0 +1,319 @@ +/* Base64 encode/decode strings or files. + Copyright (C) 2004, 2005, 2006 Free Software Foundation, Inc. + + This file is part of Base64. + + Base64 is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2, or (at your option) + any later version. + + Base64 is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS
Re: canon-host problems in cygwin
[EMAIL PROTECTED] (Eric Blake) writes: 2006-02-15 Eric Blake [EMAIL PROTECTED] * sys_socket_h.m4 (gl_HEADER_SYS_SOCKET): Don't attempt using winsock2.h or ws2tcpip.h when sys/socket.h is present. Fixes 'present but cannot compile' warnings on cygwin. * socklen.m4 (gl_TYPE_SOCKLEN_T): Use gl_HEADER_SYS_SOCKET. Don't use ws2tcpip.h if sys/socket.h works. * getaddrinfo.m4 (gl_PREREQ_GETADDRINFO): Use gl_HEADER_SYS_SOCKET. (gl_GETADDRINFO): Don't use ws2tcpip.h when sys/socket.h is present. Looks good to me, I have installed it. I didn't test it for mingw32, but I'll do that eventually. Because the getaddrinfo module would now use sys_socket directly, I also installed the patch below. Thanks! Index: modules/getaddrinfo === RCS file: /sources/gnulib/gnulib/modules/getaddrinfo,v retrieving revision 1.5 diff -u -p -r1.5 getaddrinfo --- modules/getaddrinfo 23 Sep 2005 15:48:18 - 1.5 +++ modules/getaddrinfo 16 Feb 2006 09:46:11 - @@ -14,6 +14,7 @@ gettext-h socklen stdbool strdup +sys_socket configure.ac: gl_GETADDRINFO ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: patch providing SHA-2 utilities
Jim Meyering [EMAIL PROTECTED] writes: David Madore [EMAIL PROTECTED] wrote: Here's the new patch for sha256sum and so on, against the CVS HEAD version of coreutils this time. I hope it's satisfactory. I've also sent the FSF the email for the copyright assignment. Thanks again. As you know, the papers have gone through. Hi. Would it be possible to re-license this work to the LGPL, and merge it with gnulib? I will eventually need SHA-2 in some of my projects that is using gnulib, and I want the LGPL license. I contacted FSF earlier to re-license MD5, and that was eventually done, but I'd like to avoid that experience if possible. Since this is a fairly new contribution, I hope the original authors are still responsive. Would you consider licensing your work under the LGPL? As I'm sure you realize, there's too much duplication between lib/md5.c and lib/sha*.c. Eventually, it'd be good to factor out some of that. I have a itch about this too, and eventually I will work on it. But don't let that stop anyone else from beating me to it... I think we should have a generic interface, say hash.h and hash_*, that implement all hash algorithms in a flexible way. Thanks, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: patch providing SHA-2 utilities
Eric Blake [EMAIL PROTECTED] writes: According to David Madore on 11/9/2005 9:45 AM: I believe that now that I've signed the copyright assignment to the FSF I no longer can relicense this code (at least under U.S. law: under French law I the copyright assignment is nil so I think I can, but you probably don't care much). So you'll have to ask the FSF again. My understanding (although IANAL) is that when you assign copyright to FSF, you, as original author, still retain the right to relicense your code as you see fit for other uses without involving the FSF. So the fact that you are willing to relicense means that you can put your own SHA-2 implementation under LGPL as Simon desired (I'm sure someone else will correct me if I am mis-speaking). This is my understanding as well. There is a history in gnulib to ask the authors whether they are willing to make GPL'd code available also under the LGPL, and this is often granted. For MD5, is was suggested to go through the extra step of actually asking the FSF, which later lead me to ask RMS directly, but I believe that was a special case, and not how things need to be done. David, you wrote all of the changes, right? If so, I think it would be sufficient if you would state that you release your stuff under the LGPL too. Thanks, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: bug in getaddrinfo module
[EMAIL PROTECTED] (Eric Blake) writes: gcc -DHAVE_CONFIG_H -I. -I../../lib -I.. -Wall -c ../../lib/canon-host.c ../../lib/canon-host.c: In function `canon_host_r': ../../lib/canon-host.c:71: warning: implicit declaration of function `getaddrinfo' ../../lib/canon-host.c:77: warning: implicit declaration of function `freeaddrinfo' ../../lib/canon-host.c: In function `ch_strerror': ../../lib/canon-host.c:89: warning: implicit declaration of function `gai_strerror' ../../lib/canon-host.c:89: warning: return makes pointer from integer without a cast Cygwin provides sys/socket.h and netdb.h, but has not yet provided an implementation for getaddrinfo, freeaddrinfo, or gai_strerror. It looks like getaddrinfo.h is using #ifndef HAVE_DECL_GETADDRINFO, but AC_CHECK_DECLS uses #define HAVE_DECL_GETADDRINFO 0 in config.h. Fortunately, the implicit definition of these functions works on cygwin (all arguments are int or pointer, and cygwin uses 32-bit pointers), but it is a bug waiting to happen on a 64-bit platform. Fixed thusly: Installed, thanks! ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: coreutils-5.90 build feedback
Jim Meyering [EMAIL PROTECTED] writes: On Compaq Alpha OSF/1 5.1: c89 -DHAVE_CONFIG_H -DLIBDIR=\/uufs/inscc.utah.edu/common/home/mthnhb/alpha/local/lib\ -I. -I. -I.. -I.. -I. -I/uufs/inscc.utah.edu/common/home/mthnhb/alpha/local/include -ieee -I/uufs/inscc.utah.edu/common/home/mthnhb/alpha/local/include -c getaddrinfo.c ... cc: Error: getaddrinfo.h, line 31: In this declaration, the struct addrinfo is redefined. (redefstruct) struct addrinfo ^ cc: Error: getaddrinfo.h, line 91: In this declaration, the type of gai_strerror is not compatible with the type of a previous declaration of gai_strerror at line number 293 in file /usr/include/netdb.h. (notcompat) extern const char *gai_strerror (int ecode); I'm doing a new build with cc on that system. This is happening because getaddrinfo.h defines struct addrinfo and declares gai_strerror merely because the system lacks the getaddrinfo function. But this system already has those in netdb.h, and they conflict. Simon, I think the guards should be more precise. I.e., use HAVE_GETADDRINFO only for the getaddrinfo declaration. Then test for gai_strerror and guard its declaration with HAVE_GAI_STRERROR. Similarly for struct addrinfo. Do you feel like doing that? Yes, I'll work on this. How about this patch? Not exactly like you suggested, but appears good to me. I have not tested this, but if it looks good for you, I'll install it in my projects and rebuild them, then we can sort out any remaining build problems later on. Perhaps the AI_* and EAI* define's should be guarded on a per-flag basis. Last time I checked, I didn't have simple access to any system without getaddrinfo, so I can only test this partially. If anyone know about any public build cluster without getaddrinfo, please let me know. 2005-10-01 Simon Josefsson [EMAIL PROTECTED] * getaddrinfo.m4: Use AC_GNU_SOURCE, GNU only declare getaddrinfo as an (POSIX) extension. Check for sys/types.h, sys/socket.h, and netdb.h too, needed by getaddrinfo.h. Check if getaddrinfo, freeaddrinfo and gai_strerror are declared by the POSIX headers. Check if struct addrinfo is declared. 2005-10-01 Simon Josefsson [EMAIL PROTECTED] * getaddrinfo.h: Protect #include's of sys/socket.h and netdb.h. Only define struct addrinfo if !HAVE_STRUCT_ADDRINFO. Protect AI_* and EAI_* definitions. Protect function declarations. Index: lib/getaddrinfo.h === RCS file: /cvsroot/gnulib/gnulib/lib/getaddrinfo.h,v retrieving revision 1.5 diff -u -p -r1.5 getaddrinfo.h --- lib/getaddrinfo.h 13 Sep 2005 04:15:15 - 1.5 +++ lib/getaddrinfo.h 1 Oct 2005 09:14:31 - @@ -19,13 +19,21 @@ #ifndef GETADDRINFO_H # define GETADDRINFO_H -/* Get getaddrinfo declarations, if available. Also get 'socklen_t', - and 'struct sockaddr' via sys/types.h which are used below. */ -# include sys/types.h -# include sys/socket.h -# include netdb.h +/* Get all getaddrinfo related declarations, if available. */ +# ifdef HAVE_SYS_SOCKET_H +# include sys/socket.h +# endif +# ifdef HAVE_NETDB_H +# include netdb.h +# endif + +# ifndef HAVE_STRUCT_ADDRINFO -# if !HAVE_GETADDRINFO +/* Get 'socklen_t', and 'struct sockaddr' via sys/types.h which are + used below. */ +# ifdef HAVE_SYS_TYPES_H +# include sys/types.h +# endif /* Structure to contain information about address of a service provider. */ struct addrinfo @@ -39,38 +47,47 @@ struct addrinfo char *ai_canonname; /* Canonical name for service location. */ struct addrinfo *ai_next;/* Pointer to next in list. */ }; +# endif /* Possible values for `ai_flags' field in `addrinfo' structure. */ -# define AI_PASSIVE0x0001 /* Socket address is intended for `bind'. */ -# define AI_CANONNAME 0x0002 /* Request for canonical name. */ -# define AI_NUMERICHOST0x0004 /* Don't use name resolution. */ -# define AI_V4MAPPED 0x0008 /* IPv4 mapped addresses are acceptable. */ -# define AI_ALL0x0010 /* Return IPv4 mapped and IPv6 addresses. */ -# define AI_ADDRCONFIG 0x0020 /* Use configuration of this host to choose +# ifndef AI_PASSIVE +# define AI_PASSIVE 0x0001 /* Socket address is intended for `bind'. */ +# define AI_CANONNAME 0x0002 /* Request for canonical name. */ +# define AI_NUMERICHOST 0x0004/* Don't use name resolution. */ +# define AI_V4MAPPED 0x0008 /* IPv4 mapped addresses are acceptable. */ +# define AI_ALL 0x0010 /* Return IPv4 mapped and IPv6 addresses. */ +# define AI_ADDRCONFIG0x0020 /* Use configuration of this host to choose returned address type.. */ +# endif /* Error values for `getaddrinfo' function. */ -# define EAI_BADFLAGS-1/* Invalid value for `ai_flags' field
Re: gethostbyname on Solaris 5.7
Jim Meyering [EMAIL PROTECTED] writes: Here's a log of a failed Solaris 5.7 build cc -I/usr/local/include -R/usr/local/lib -L/usr/local/lib -o pinky pinky.o ../lib/libcoreutils.a /usr/local/lib/libintl.so /usr/local/lib/libiconv.so -lc -R/usr/local/lib ../lib/libcoreutils.a Undefined first referenced symbol in file gethostbyname ../lib/libcoreutils.a(getaddrinfo.o) getservbyname ../lib/libcoreutils.a(getaddrinfo.o) ld: fatal: Symbol referencing errors. No output written to pinky make[3]: *** [pinky] Error 1 The problem was that coreutils no longer had the check for -lnsl. coreutils-5.3.0 had that test in canon-host.m4, and now that canon-host is just a wrapper around getaddrinfo, it's appropriate that it no longer perform this check. So getaddrinfo.m4 must do it, now, and I've made the following change in coreutils, and will commit it to gnulib as soon as I've confirmed it works. Hi! Looks good, thanks! 2005-09-24 Jim Meyering [EMAIL PROTECTED] * getaddrinfo.m4 (gl_GETADDRINFO): Check for gethostbyname in the inet and nsl libraries. Required on Solaris 5.7. Index: m4/getaddrinfo.m4 === RCS file: /fetish/cu/m4/getaddrinfo.m4,v retrieving revision 1.4 retrieving revision 1.5 diff -u -p -u -r1.4 -r1.5 --- m4/getaddrinfo.m4 23 Sep 2005 19:40:04 - 1.4 +++ m4/getaddrinfo.m4 24 Sep 2005 10:44:30 - 1.5 @@ -1,4 +1,4 @@ -# getaddrinfo.m4 serial 4 +# getaddrinfo.m4 serial 5 dnl Copyright (C) 2004, 2005 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -7,6 +7,7 @@ dnl with or without modifications, as lo AC_DEFUN([gl_GETADDRINFO], [ AC_SEARCH_LIBS(getaddrinfo, nsl socket) + AC_SEARCH_LIBS(gethostbyname, [inet nsl]) AC_REPLACE_FUNCS(getaddrinfo gai_strerror) gl_PREREQ_GETADDRINFO ]) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Eric Blake [EMAIL PROTECTED] writes: According to Simon Josefsson on 6/27/2005 3:55 PM: -w is for encoding, -i for decoding. So they are orthogonal. I thought about making -w affect decoding to, so decoding would ignore (only) newlines after COLS. Could be fixed later on. I would have suggested this as well; I like the thought of -w during encoding adds newline at COL and at EOF, and -w during decoding ignores newline only at COL and EOF. Yes. As to your questions on other command line interfaces, I think base64 should at least behave as a filter: base64 file file.b64 base64 -d file.b64 file Yup. This is possible now. As for whether choosing on base64 abc whether abc is the filename containing text to be encoded, or the actual text to be encoded, I would lean towards filenames. Me too. The tool started out as a debugging tool for me, but it is not the typical use. And 'base64 -s foo' is simple for short tests. Or should it be 'base64 -e foo'? To match 'grep -e ...' or 'perl -e ...'. 'foo' is a string here, not a filename. Consider that your typical input is arbitrary length binary data, and passing that binary data directly as a command line argument is not a typical usage of a textual command line interface. Besides, if you can easily type the input as an ASCII argument, why do you need the 133% size penalty of encoding it? For a single file, it might make sense to support a -o/--output option to specify the name of the output file. Then base64 would be equivalent to base64 - -o -. I'm not sure of the best approach for scaling it to multiple input or output files, or if that is even necessary. Dealing with multiple input files could also be done later on, I think it is more important to make sure the arguments are filenames and not strings now. Does the RFC permit, recommend, or discourage \r\n line endings in base64 encodings? Since the output is printable, you may want to consider carefully the decision between opening the coded file in text vs. binary modes on platforms where they are different. The decoded file must definitely be opened binary. The RFC is fuzzy on the issue, but I want to discourage too lax treatment of encoded data. Lax treatment create opportunities for side channel attacks, and make implementations complex since they have to deal with various ill-formed input. So without --wrap, I definitely think the file should be opened as binary. But with wrap it is less clear. The RFC doesn't discuss what newline means. So I think we should open it as text. More thoughts appreciated. Cheers, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
[EMAIL PROTECTED] (Eric Blake) writes: The RFC is fuzzy on the issue, but I want to discourage too lax treatment of encoded data. Lax treatment create opportunities for side channel attacks, and make implementations complex since they have to deal with various ill-formed input. So without --wrap, I definitely think the file should be opened as binary. But with wrap it is less clear. The RFC doesn't discuss what newline means. So I think we should open it as text. More thoughts appreciated. One thing to remember is that many RFCs expect \r\n (consider SMTP, HTTP, etc), so it might be nice to allow that even on Unix systems where only \n is normal! Maybe it is worth a command-line argument, where on encoding you can change the default line-ending (this requires opening in binary mode to guarantee that the OS doesn't change your wishes), and on decoding you can choose whether to be lax on any line-ending or robust to only allow one type of line-ending (again, this implies binary mode). But it's your call (esp. since you are the editor of RFC 3548 :). How about -l/--line-ending ENDING where ending is 'crlf', 'cr', 'lf', or 'any', and omitting -l or using -lany on encode defaults to lf, and omitting -l on decode defaults to any. I wish I could avoid falling into the EOL-sandpit, it appear complex and error prone... I'll try to get the tool up and running according to the other comments first, and then revisit this issue when things are more clear code-wise. Don't let that stop you from thinking about this, there may be other ideas worth investigating. Thanks, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Paul Eggert [EMAIL PROTECTED] writes:
[EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] -w
[EMAIL PROTECTED] --wrap
[EMAIL PROTECTED] wrap data
[EMAIL PROTECTED] column to wrap data after
+During encoding, wrap lines after @var{COLS} characters. This must be
+a positive number.
+
+If this option is not given at all, no line wrapping will be
+performed. If @var{COLS} is omitted, the default is 76.
The POSIX utility syntax guidelines do not allow options with optional
arguments, and I tend to agree, at least for single-letter options.
How about if we simply require COLS?
Yup, Eric Blake's suggestion was much better. Make the default to
always wrap after 76, require COLS, have 0 mean no wrapping.
[EMAIL PROTECTED] -i
[EMAIL PROTECTED] --ignore-garbage
[EMAIL PROTECTED] -i
[EMAIL PROTECTED] --ignore-garbage
[EMAIL PROTECTED] Ignore garbage in base64 stream
+During decoding, ignore unrecognized characters (including newline),
+to permit distorted data to be decoded.
Is this option needed if the encoder used the -w option? It's
not clear from the documentation.
-w is for encoding, -i for decoding. So they are orthogonal. I
thought about making -w affect decoding to, so decoding would ignore
(only) newlines after COLS. Could be fixed later on.
+ size_t * outlen)
* outlen - *outlen
Fixed, ran indent too.
+ const char b64[64] =
+ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/;
Surely this should be static, not auto.
Yup.
Also, it's a bit tricky that there's a local var b64 and a global
one also named b64. I'd rename one of them.
The local one is now 'b64str'.
+ *out++ = b64[((to_uchar (in[0]) 4)
++ (--inlen ? to_uchar (in[1]) 4 : 0)) 0x3f];
That 0x3f is hard to follow. Here's a better and more-consistent
indenting style:
*out++ = b64[((to_uchar (in[0]) 4)
+ (--inlen ? to_uchar (in[1]) 4 : 0))
0x3f];
+ *out++ =
+ (inlen
+ ? b64[((to_uchar (in[1]) 2)
+ + (--inlen ? to_uchar (in[2]) 6 : 0)) 0x3f] : '=');
Similarly, the 0x3f and : '=' are indented confusingly. How
about this instead?
*out++ =
(inlen
? b64[((to_uchar (in[1]) 2)
+ (--inlen ? to_uchar (in[2]) 6 : 0))
0x3f]
: '=');
Yup.
+ if (!--outlen)
+break;
This is in a loop that will test outlen anyway. Why is
the if-test needed here? Can't you simply decrement outlen?
Which of the if statements are you referring to? Outlen is decremented
several times in the loop, and can become zero at any one of the
conditioned decrements, so has to be checked after every decrement to
avoid that the next decrement wrap the counter.
+ in += 3;
This assignment might have undefined behavior if IN points at or near
the end of a buffer. The C standard says you can't point more than
one byte past the end of a buffer.
Does undefined behavior include crashing? Or merely that the
variable may contain an undefined value? If the latter, that
shouldn't be a problem.
In any case, I think it could be guarded as:
if (inlen)
in += 3;
since inlen would be 0 at that point if we are near the end of the
buffer.
+The data is encoded as described for the base64 alphabet in RFC 3548.\n\
data is - data are
(Data is a plural in English.)
Thanks!
+ n = fread (inbuf + sum, 1, BLOCKSIZE - sum, in);
+
+ if (n BLOCKSIZE - sum)
+error (EXIT_FAILURE, 0, _(read too much));
...
+ if (n B64BLOCKSIZE - sum)
+error (EXIT_FAILURE, 0, _(read too much));
That's not possible. If you want to test for this, you should simply
abort () if you read too much. But I wouldn't bother testing for this
(the rest of coreutils doesn't).
Ok, tests removed.
+ /* When wrapping, terminate last line. */
+ if (wrap_column fputs (\n, out) 0)
+error (EXIT_FAILURE, errno, _(write error));
Might this output a newline that is adjacent to a previous newline?
Is the double-newline intended?
I'm going to rewrite the wrapping logic, so I'll postpone that one.
+ if (ferror (in))
+error (EXIT_FAILURE, errno, _(read error));
It's not correct to use errno here, since it may have been set by
a previous system call to something other than the errno that
corresponds to the read error. (There are two instances
of this problem.)
I'll check this problem too after the rewrite.
+ if (ignore_garbage)
+{
+ size_t i;
+ for (i = 0; n 0 i n;)
+if (isbase64 (inbuf[sum + i]) || inbuf[sum + i] == '=')
+ i++;
+else
+ memmove (inbuf + sum + i, inbuf + sum + i + 1, --n - i);
+}
That's an O(N**2) algorithm. Why not use an O(N) algorithm instead?
You can do that by copying each valid byte of the buffer as you go.
You can do
Re: base64 tool?
Jim Meyering [EMAIL PROTECTED] writes: Is a base64 encode/decode tool suitable for coreutils? As Padraig noted, GNU recode does this: $ echo foo|recode ../64 Zm9vCg== $ echo foo|recode ../64|recode /64 foo but I find the syntax to be obscure. And it's not always available. Agreed. Since last time this was discussed, I have released a couple of versions of base64 http://josefsson.org/base64/. It is stable, but I suppose there are not many users yet. I asked RMS whether it could be part of the GNU project, and he agreed, but also thought that because of its tiny size, it would be nice to bundle it together with some other package. Since base64 rely heavily on gnulib modules, and the tool was strongly influenced by the coding style of several coreutils tools, making base64 be part of coreutils seem to be a good idea, IMHO. What do you think? I understand accepting new tools into coreutils is probably done rather conservatively. But base64 encoding is pretty common these days. I'll volunteer to submit a patch and documentation for the tool if you think it could be added. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Jim Meyering [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] wrote: ... I'll volunteer to submit a patch and documentation for the tool if you think it could be added. Ok. That sounds fair. If you can add some tests too, that'd be great. If so, please use something like coreutils/tests/md5sum/basic-1 as a model. Great! My package contain several self tests, they can probably be adapted easily. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Simon Josefsson [EMAIL PROTECTED] writes: Jim Meyering [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] wrote: ... I'll volunteer to submit a patch and documentation for the tool if you think it could be added. Ok. That sounds fair. If you can add some tests too, that'd be great. If so, please use something like coreutils/tests/md5sum/basic-1 as a model. Great! My package contain several self tests, they can probably be adapted easily. Initial patch below; review is especially needed for the coreutils.texi part. Self tests after lunch... /ChangeLog: 2005-06-25 Simon Josefsson [EMAIL PROTECTED] * AUTHORS: Mention base64. * NEWS: Likewise. * man/Makefile.am: Build base64.1. * man/base64.x: New file. * src/Makefile.am (bin_PROGRAMS): Add base64. * src/base64.c: New file. doc/ChangeLog: 2005-06-25 Simon Josefsson [EMAIL PROTECTED] * coreutils.texi: Add base64 section. lib/ChangeLog: 2005-06-25 Simon Josefsson [EMAIL PROTECTED] * base64.h, base64.c: New files. * Makefile.am (libcoreutils_a_SOURCES): Add base64.h and base64.c. Index: AUTHORS === RCS file: /cvsroot/coreutils/coreutils/AUTHORS,v retrieving revision 1.7 diff -u -p -r1.7 AUTHORS --- AUTHORS 3 Nov 2004 23:10:50 - 1.7 +++ AUTHORS 25 Jun 2005 13:28:38 - @@ -1,6 +1,7 @@ Here are the names of the programs in this package, each followed by the name(s) of its author(s). +base64: Simon Josefsson basename: FIXME unknown cat: Torbjorn Granlund, Richard M. Stallman chgrp: David MacKenzie, Jim Meyering Index: NEWS === RCS file: /cvsroot/coreutils/coreutils/NEWS,v retrieving revision 1.299 diff -u -p -r1.299 NEWS --- NEWS23 Jun 2005 22:37:33 - 1.299 +++ NEWS25 Jun 2005 13:28:40 - @@ -167,6 +167,11 @@ GNU coreutils NEWS stat -f's default output format has been changed to output this size as well. stat -f recognizes file systems of type XFS and JFS +** New programs + + base64 is a new tool that provide base64 encoding and decoding + functionality. + * Major changes in release 5.3.0 (2005-01-08) [unstable] ** Bug fixes Index: README === RCS file: /cvsroot/coreutils/coreutils/README,v retrieving revision 1.18 diff -u -p -r1.18 README --- README 8 Jan 2005 09:00:08 - 1.18 +++ README 25 Jun 2005 13:28:40 - @@ -7,13 +7,14 @@ arbitrary limits. The programs that can be built with this package are: - [ basename cat chgrp chmod chown chroot cksum comm cp csplit cut date dd - df dir dircolors dirname du echo env expand expr factor false fmt fold - ginstall groups head hostid hostname id join kill link ln logname ls - md5sum mkdir mkfifo mknod mv nice nl nohup od paste pathchk pinky pr - printenv printf ptx pwd readlink rm rmdir seq sha1sum shred sleep sort - split stat stty su sum sync tac tail tee test touch tr true tsort tty - uname unexpand uniq unlink uptime users vdir wc who whoami yes + [ base64 basename cat chgrp chmod chown chroot cksum comm cp csplit + cut date dd df dir dircolors dirname du echo env expand expr factor + false fmt fold ginstall groups head hostid hostname id join kill + link ln logname ls md5sum mkdir mkfifo mknod mv nice nl nohup od + paste pathchk pinky pr printenv printf ptx pwd readlink rm rmdir seq + sha1sum shred sleep sort split stat stty su sum sync tac tail tee + test touch tr true tsort tty uname unexpand uniq unlink uptime users + vdir wc who whoami yes See the file NEWS for a list of major changes in the current release. Index: doc/coreutils.texi === RCS file: /cvsroot/coreutils/coreutils/doc/coreutils.texi,v retrieving revision 1.269 diff -u -p -r1.269 coreutils.texi --- doc/coreutils.texi 24 Jun 2005 08:05:51 - 1.269 +++ doc/coreutils.texi 25 Jun 2005 13:28:44 - @@ -36,6 +36,7 @@ @dircategory Individual utilities @direntry +* base64: (coreutils)base64 invocation. Base64 encode/decode data. * basename: (coreutils)basename invocation. Strip directory and suffix. * cat: (coreutils)cat invocation. Concatenate and write files. * chgrp: (coreutils)chgrp invocation. Change file groups. @@ -217,6 +218,7 @@ Output of entire files * tac invocation:: Concatenate and write files in reverse. * nl invocation:: Number lines and write files. * od invocation:: Write files in octal or other formats. +* base64 invocation:: Transform data into printable data. Formatting file contents @@ -1183,6 +1185,7 @@ in some way. * tac invocation:: Concatenate and write files in reverse. * nl invocation
Re: base64 tool?
Eric Blake [EMAIL PROTECTED] writes:
s/provide/provides/
s/transform/transforms/
s/expand/expands/
Fixed, thanks! I really need a flygrammar.el. Or some english
classes.
[EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] -w
[EMAIL PROTECTED] --wrap
[EMAIL PROTECTED] wrap data
[EMAIL PROTECTED] column to wrap data after
+During encoding, wrap lines after @var{COLS} characters. This must be
+a positive number.
+
+If this option is not given at all, no line wrapping will be
+performed. If @var{COLS} is omitted, the default is 76.
Do we really want optional arguments to short options? POSIX tends to
prefer required arguments, so that -w76 and -w 76 behave the same. And if
base64 is ever adopted by a future version of POSIX, we might as well be
ready for it.
Also, I wonder if a default of 76 when --wrap is not specified or
specified without argument, and --wrap=0 as the special case to disable
wrapping, is more likely to be useful, since printable data usually
implies useful line wraps.
All good ideas, I think. I'll change it so -w/--wrap always take an
argument, and 76 is the default, and 0 disable wrapping.
Is -i valid during encoding, or does your synopsis need to be updated to
show that -i can only accompany -d?
The latter.
More thoughts on the command line interface:
I think there should be a -s/--string argument that is used when you
want to encode/decode a specific string, i.e. instead of:
base64 foo
base64 --decode foo
you type
base64 -s foo
base64 --decode --string foo
This would allow the following:
base64 myfile.txt
to read a file given as an argument.
However, where should the output go?
'base64 myfile.txt' could create output in myfile.txt.b64. Then
base64 will work similar to gzip. 'base64 -c myfile.txt' could be
added to print to stdout. And possibly 'base64 myfile.txt myfile.b64'
could be supported, to specify INFILE and OUTFILE. Or 'base64
myfile.txt anotherfile.txt' could produce myfile.txt.b64 and
anotherfile.txt.b64.
Alternatively, 'base64 myfile.txt' could produce output on standard
output. Then you could possibly support 'base64 myfile1.txt
myfile2.txt myfile3.txt' which would concatenate the input and base64
encode it all. I don't think this mode is very useful. You can't
decode the data into separate files again. And if this mode is ever
useful, it is easy to work around by 'cat myfile1.txt myfile2.txt
myfile3.txt|base64'.
More thoughts on the command line syntax are appreciated.
Thanks,
Simon
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: base64 tool?
Philip Rowlands [EMAIL PROTECTED] writes: There's the base64 openssl applet, which can use argv[0]: $ ln -s /usr/bin/openssl base64 $ echo This is a test | ./base64 VGhpcyBpcyBhIHRlc3QK $ echo VGhpcyBpcyBhIHRlc3QK | openssl base64 -d This is a test Right, although it doesn't have customizable line wrapping. Nor is it as standalone as most of the coreutils tools are... Jim Meyering [EMAIL PROTECTED] writes: Is a base64 encode/decode tool suitable for coreutils? As Padraig noted, GNU recode does this: $ echo foo|recode ../64 Zm9vCg== $ echo foo|recode ../64|recode /64 foo but I find the syntax to be obscure. And it's not always available. Agreed. FWIW, I have written a base64 tool to satisfy my needs. It is based on the coreutils code style. If at some point in time, the need for a base64 tool in coreutils arise, it should not be difficult to adapt my implementation. http://josefsson.org/base64/ http://josefsson.org/cgi-bin/viewcvs.cgi/base64/src/base64.c?view=markup Thanks, Simon ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
base64 tool?
Is a base64 encode/decode tool suitable for coreutils? I typically use `M-: (base64-encode-string foo)' in Emacs, but I have found I often want a command line tool as well. And `echo foo|base64' is easier to type. There is a base64 module in gnulib that I think would be suitable for this. It should probably not read in the entire file in memory, though. I'm thinking: Usage: base64 [OPTION] [FILE]... Encode binary data using printable characters. With no FILE, or when FILE is -, read standard input. Optional arguments to long options are optional for short options too. -d, --decodedecode data -w, --wrap[=COLS] wrap encoded lines after COLS character, default 76. --ignore-garbagewhen decoding, ignore non-alphabet characters --help display this help and exit --version output version information and exit The data is encoded as described for the base64 alphabet in RFC 3548. Decoding require compliant input by default, use --ignore-garbage to attempt to recover from non-alphabet characters (such as newlines) in the encoded stream. Report bugs to bug-coreutils@gnu.org. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
announce-gen perl 5.8.1 bug fix
Index: announce-gen
===
RCS file: /cvsroot/coreutils/coreutils/announce-gen,v
retrieving revision 1.17
diff -u -p -u -w -r1.17 announce-gen
--- announce-gen26 Aug 2003 07:19:57 - 1.17
+++ announce-gen2 Oct 2003 19:34:36 -
@@ -175,7 +175,7 @@ sub print_changelog_deltas ($$)
'current-version=s' = \$curr_version,
'release-archive-directory=s' = \$release_archive_dir,
'url-directory=s@' = [EMAIL PROTECTED],
- 'news=s@' = \$news_file,
+ 'news=s' = \$news_file,
help = sub { usage 0 },
version = sub { print $ME version $VERSION\n; exit },
___
Bug-coreutils mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/bug-coreutils
Re: make check fix
Paul Eggert [EMAIL PROTECTED] writes: Simon Josefsson [EMAIL PROTECTED] writes: For some reason sort in LANG=sv_SE.UTF-8 sort 'w' equal to 'v', making 'wc' end up before 'vdir', and a check fails. Thanks for pointing this out. But the patch isn't quite right, since you can't portably set LC_COLLATE to C without also setting LC_CTYPE consistently, as the resulting behavior is undefined. It's simpler just to set LC_ALL. Also, there are some other places that have the same problem. How about the following patch instead? Looks better. Thanks! ___ Bug-coreutils mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/bug-coreutils
