Re: [RFC] malloced getpw/grxxx functions for bb

2014-12-06 Thread tito
On Sunday 09 November 2014 20:56:07 tito wrote:
> On Sunday 28 September 2014 15:24:50 tito wrote:
> > On Saturday 27 September 2014 14:58:08 tito wrote:
> > > On Wednesday 24 September 2014 23:02:55 tito wrote:
> > > > On Saturday 20 September 2014 16:32:01 tito wrote:
> > > > > Hi,
> > > > > One more fix of a return value.
> > > > > 
> > > > > Ciao,
> > > > > Tito
> > > > > 
> > > > 
> > > > Hi,
> > > > more return value and errno fixes.
> > > > 
> > > > Ciao,
> > > > Tito
> > > > 
> > > Hi,
> > > make the tokenize function more robust.
> > > 
> > > Ciao,
> > > Tito
> > > 
> > Hi,
> > more minor errno fixes and code cleanups.
> > 
> > Ciao,
> > Tito
> > 
> 
> Hi,
> is there anybody bold out there that wants to review
> this stuff?
> 
> Ciao,
> Tito
> 

Hi,
make the  parse_common function more robust.

Ciao,
Tito
/* vi: set sw=4 ts=4: */
/* Copyright (C) 2003 Manuel Novoa III
 *
 * Licensed under GPLv2 or later, see file LICENSE in this source tree.
 */

/* Nov 6, 2003  Initial version.
 *
 * NOTE: This implementation is quite strict about requiring all
 *field seperators.  It also does not allow leading whitespace
 *except when processing the numeric fields.  glibc is more
 *lenient.  See the various glibc difference comments below.
 *
 * TODO:
 *Move to dynamic allocation of (currently statically allocated)
 *  buffers; especially for the group-related functions since
 *  large group member lists will cause error returns.
 */

/* Copyright (C) 2014   Tito Ragusa 
 * 
 * Licensed under GPLv2 or later, see file LICENSE in this source tree.
 */

/* Rewrite of some parts. Main differences are:
 * 
 * 1) the buffer for getpwuid, getgrgid, getpwnam, getgrnam is dynamically
 *allocated and reused by later calls. if ERANGE error pops up it is
 *reallocated to the size of the longest line found in the passwd/group
 *files and reused for later calls.
 *
 * 2) the passwd/group files:
 *  a) must contain the expected number of fields;
 *  b) some fields are not allowed to be empty (e.g. username, homedir, shell);
 *  c) leading or trailing spaces in fields are not allowed;
 *  d) the string representing uid/gid must be convertible by strtoXX functions;
 *if the above explained conditions are not met a error message about bad
 *record in file is printed so that the user knows about it.
 * 3) the internal function for getgrouplist uses a dynamically allocated
 *buffer and retries with a bigger one in case it is to small;
 * 4) the _r functions use the user supplied buffers that are never reallocated
 *but use mostly the same common code as the other functions.
 * 5) at the moment only the functions really used by busybox code are
 *implemented, if you need a particular missing function it should be
 *easy to write it by using the internal common code.
 */

#include "libbb.h"

/* for the fields in passwd/group/shadow files */
#define NOT_EMPTY 0
#define MAYBE_EMPTY 1

/* Number of fields in passwd file starting from 1 */
#define _PASSWD_FIELDS 7
/* S = string not empty, s = string maybe empty,  */
/* I = uid,gid, l = long maybe empty, m = members */
static const char *pw_def = "SsIIsSS";

/* Number of fields in group file starting from 1 */
#define _GROUP_FIELDS  4
static const char *gr_def = "SsIm";

/* Number of fields in shadow file starting from 1 */
#define _SHADOW_FIELDS 9
#if ENABLE_USE_BB_SHADOW
/* But we parse only 8 as last is reserved*/
static const char *sp_def = "Ssll";
#endif

/* Initial buffer size */
#define PWD_GRP_BUFSIZE256
/* for getpw_common_malloc */
static char *pwd_buffer = NULL;
static size_t pwd_buffer_size = PWD_GRP_BUFSIZE;
struct passwd *my_pw;
struct passwd my_pwd;
/* for getgr_common_malloc */
static char *grp_buffer = NULL;
static size_t grp_buffer_size = PWD_GRP_BUFSIZE;
struct group *my_gr;
struct group my_grp;
/* for setpwent, getpwent_r, endpwent */
static FILE *pwf = NULL;
/* for setgrent, getgrent_r, endgrent */
static FILE *grf = NULL;

/* TODO: We do no file locking, but we do
 * in libbb/update_passwd.c, so maybe the same code
 * could be reused here */
#define LOCK		((void) 0)
#define UNLOCK		((void) 0)

/**/
/* Internal functions */
/**/

/* Divide the passwd/group/shadow record in fields
 * by substituting the given delimeter
 * e.g. ':' or ',' with '\0'.
 * Returns the  number of fields found.
 * Checks for leading or trailing spaces in fields.
 */
static int FAST_FUNC tokenize(char *buffer, int ch, int *error)
{
	char *p = buffer;
	char *s = p;
	int num_fields = 1;

	while (*p) {
		if (*p == ch) {
			*p = '\0';
			num_fields++;
			s = p + 1;
		}
		if (*s == ' ' || last_char_is(s, ' ') != NULL) {
			*error = EINVAL;
		}
		p++;
	}
	return num_fields;
}

/* Fill the buffer linebuf up to len with
 * a line of 

Re: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

2014-12-06 Thread Denys Vlasenko
Please see attached patch (only compile-tested so far).
It builds on your work, implementing in-memory compression
for all unpackers we have.

Unfortunately, bloatcheck is not inspiring:

function old new   delta
transformer_write  - 133+133
setup_transformer_on_fd- 133+133
complete_cmd_dir_file773 895+122
open_transformer_state -  92 +92
xmalloc_open_zipped_read_close63 112 +49
xtransformer_write -  24 +24
unpack_lzma_stream  27172737 +20
unpack_xz_stream23932407 +14
unpack_Z_stream 11731183 +10
unpack_gz_stream 693 696  +3
decode_one_format726 729  +3
refresh  665 667  +2
inflate_unzip111 113  +2
unpack_bz2_stream359 360  +1
swap_on_off_main 420 418  -2
scan_recursive   380 378  -2
inflate_unzip_internal  19451924 -21
open_zipped   89  47 -42
setup_unzip_on_fd142  53 -89
--
(add/remove: 4/0 grow/shrink: 10/5 up/down: 608/-156) Total: 452 bytes
   textdata bss dec hex filename
 923218 928   17684  941830   e5f06 busybox_old
 923686 928   17684  942298   e60da busybox_unstripped
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox


Re: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

2014-12-06 Thread Denys Vlasenko
On Sat, Dec 6, 2014 at 5:11 PM, Denys Vlasenko  wrote:
> Please see attached patch (only compile-tested so far).
> It builds on your work, implementing in-memory compression
> for all unpackers we have.


1.patch
Description: Binary data
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

2014-12-06 Thread Lauri Kasanen
On Sat, Dec 6, 2014, at 18:11, Denys Vlasenko wrote:
> Please see attached patch (only compile-tested so far).
> It builds on your work, implementing in-memory compression
> for all unpackers we have.

It misses the "plus one zero byte" behavior of the non-compressed path
(xmalloc_read). I bet some callers depend on it, using str* functions on
the returned buffer.

- Lauri

-- 
http://www.fastmail.com - Same, same, but different...

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox


Re: [PATCH] Add a gzip fastpath for the xmalloc readers, v3

2014-12-06 Thread Denys Vlasenko
On Sat, Dec 6, 2014 at 7:41 PM, Lauri Kasanen  wrote:
> On Sat, Dec 6, 2014, at 18:11, Denys Vlasenko wrote:
>> Please see attached patch (only compile-tested so far).
>> It builds on your work, implementing in-memory compression
>> for all unpackers we have.
>
> It misses the "plus one zero byte" behavior of the non-compressed path
> (xmalloc_read). I bet some callers depend on it, using str* functions on
> the returned buffer.

Fixed, and committed to git. Please try it.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox