Re: [PATCH] Fix const pointer assignment trick on clang 9+
Tried, not working.
On Fri, Oct 8, 2021 at 10:21 PM Denys Vlasenko wrote:
>
> On Fri, Oct 8, 2021 at 10:39 AM 余生与君 wrote:
> > On Thu, Oct 7, 2021 at 9:48 PM Denys Vlasenko
> > wrote:
> > > Can you try something more? E.g. (in current git):
> > >
> > > #define ASSIGN_CONST_PTR(p, v) do { \
> > > *(void**)not_const_pp() = (void*)(v); \
> > > /* At least gcc 3.4.6 on mipsel needs optimization barrier */ \
> > > barrier(); \
> > > + sleep(0); \
> > > } while (0)
> >
> > Cool! Sleep(0) does the magic!
> >
> > ADRPX21, #ash_ptr_to_globals_misc_ptr@PAGE
> > LDR X21, [X21,#ash_ptr_to_globals_misc_ptr@PAGEOFF]
> > MOV X8, X21
> > STR X0, [X8]
> > MOV W0, WZR
> > BL sleep
> > LDR X21, [X21]
> >
> >
> > And further investigation shows that a dummy function can also do this
> > trick!
> >
> > hack.c:
> > void clang_barrier() {
> > }
>
> We don't need the function call. We only need to tell clang that
> a lot of registers are "needed" for something.
> This "something" ideally should be nothing.
>
> Can you try this?
>
> #if defined(__clang_major__) && __clang_major__ >= 9
> /* Clang/llvm drops assignment to "constant" storage. Silently.
> * Needs serious convincing to not eliminate the store.
> */
> static ALWAYS_INLINE void register_barrier(void)
> {
> int r0, r1, r2, r3, r4, r5;
> asm volatile (
> "# clobber 6 registers"
> : "=r"(r0), "=r"(r1), "=r"(r2), "=r"(r3), "=r"(r4), "=r"(r5)
> : "0"(r1), "1"(r1), "2"(r2), "3"(r3), "4"(r4), "5"(r5)
> : "memory"
>
> );
> }
> static ALWAYS_INLINE void* not_const_pp(const void *p)
> {
> void *pp;
> asm volatile (
> "# forget that p points to const"
> : /*outputs*/ "=r" (pp)
> : /*inputs*/ "0" (p)
> );
> return pp;
> }
> #define ASSIGN_CONST_PTR(p, v) do { \
> *(void**)not_const_pp() = (void*)(v); \
> register_barrier();
> } while (0)
> #else
> static ALWAYS_INLINE void* not_const_pp(const void *p) { return (void*)p; }
> #define ASSIGN_CONST_PTR(p, v) do { \
> *(void**)not_const_pp() = (void*)(v); \
> /* At least gcc 3.4.6 on mipsel needs optimization barrier */ \
> barrier(); \
> } while (0)
> #endif
>
> If this works, experiment what minimum number of rN
> in register_barrier() is actually needed to placate clang.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] Fix const pointer assignment trick on clang 9+
Tried. Not working.
On Fri, Oct 8, 2021 at 10:35 PM Henrique de Moraes Holschuh
wrote:
>
> On 08/10/2021 05:39, 余生与君 wrote:
> > And further investigation shows that a dummy function can also do this
> > trick!
>
> When one wants any sort of barrier (compiler optimization barrier, CPU
> execution barrier, memory barriers, etc) that works (and is also
> extremely likely to remain working in the future) for both GCC and
> CLANG, it is usually a good idea to take a look at what the linux kernel
> is using.
>
>
> --8<-- from include/linux/compiler.h (latest linux kernel source)
>
> /* Optimization barrier */
> #ifndef barrier
> /* The "volatile" is due to gcc bugs */
> # define barrier() __asm__ __volatile__("": : :"memory")
> #endif
>
> #ifndef barrier_data
> /*
> * This version is i.e. to prevent dead stores elimination on @ptr
> * where gcc and llvm may behave differently when otherwise using
> * normal barrier(): while gcc behavior gets along with a normal
> * barrier(), llvm needs an explicit input variable to be assumed
> * clobbered. The issue is as follows: while the inline asm might
> * access any memory it wants, the compiler could have fit all of
> * @ptr into memory registers instead, and since @ptr never escaped
> * from that, it proved that the inline asm wasn't touching any of
> * it. This version works well with both compilers, i.e. we're telling
> * the compiler that the inline asm absolutely may see the contents
> * of @ptr. See also: https://llvm.org/bugs/show_bug.cgi?id=15495
> */
> # define barrier_data(ptr) __asm__ __volatile__("": :"r"(ptr) :"memory")
> #endif
>
> --8<--
>
> --
> Henrique de Moraes Holschuh
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash: improve speed of variable pattern substitution
On Wed, Jul 21, 2021 at 10:54 PM Alin Mr wrote:
>
> Ron, thanks for your quick patch and the hints. I saw your message and was
> thinking through; fixed-length patterns don't need backtracking. '?'
> (without '*') also results in fixed-length patterns, so maybe that'd be more
> complicated to detect.
>
> To motivate this a bit: it started with quoters. Since we have neither bash's
> ${x@Q}, nor `printf %q`, it's harder to escape shell code for eval, or to
> produce quoted SQL values **efficiently**.
A patch to implement e.g. ${x@Q} is acceptable.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] ash.c: speedup ${s:} substring (no quotes)
On Wed, Jul 28, 2021 at 10:40 AM Alin Mr wrote:
> This trivial patch makes ${s:...} at least as fast as ${s#??..} in
> simple tests. It's probably faster for longer substrings, but then one
> wouldn't use ${s#"1024???s"} anyway -- one would switch away from sh.
> memmove could possibly be replaced by memcpy.
>
> I haven't found much on testing (or contributing for that matter) on the
> website or in the source. `make test` produces a bunch of failures on
> master anyway, so I'm not sure how to test new code.
cd shell/ash_test
./run-all
> Signed-off-by: Alin Mr
> ---
> shell/ash.c | 23 +--
> 1 file changed, 17 insertions(+), 6 deletions(-)
>
> diff --git a/shell/ash.c b/shell/ash.c
> index b5947147a..df4d82e39 100644
> --- a/shell/ash.c
> +++ b/shell/ash.c
> @@ -7198,14 +7198,25 @@ subevalvar(char *start, char *str, int strloc,
> if ((unsigned)len > (orig_len - pos))
> len = orig_len - pos;
>
> - for (vstr = startp; pos; vstr++, pos--) {
> - if (quotes && (unsigned char)*vstr == CTLESC)
> - vstr++;
> + if (! quotes) {
> + vstr = startp + pos;
> + pos = 0;
"pos = 0" is useless
> + } else {
> + for (vstr = startp; pos; vstr++, pos--) {
> + if ((unsigned char)*vstr == CTLESC)
> + vstr++;
> + }
> }
> - for (loc = startp; len; len--) {
> - if (quotes && (unsigned char)*vstr == CTLESC)
> + if (! quotes) {
You can also merge if (!quotes) {} blocks - why test the condition twice?
> + memmove (loc, vstr, len);
Should be memmove(startp, vstr, len).
> + loc = startp + len;
> + len = 0;
"len = 0" is useless.
> + } else {
> + for (loc = startp; len; len--) {
> + if ((unsigned char)*vstr == CTLESC)
> + *loc++ = *vstr++;
> *loc++ = *vstr++;
> - *loc++ = *vstr++;
> + }
> }
> *loc = '\0';
> goto out;
Applied in this form:
if (!quotes) {
loc = mempcpy(startp, startp + pos, len);
} else {
for (vstr = startp; pos != 0; pos--) {
if ((unsigned char)*vstr == CTLESC)
vstr++;
vstr++;
}
for (loc = startp; len != 0; len--) {
if ((unsigned char)*vstr == CTLESC)
*loc++ = *vstr++;
*loc++ = *vstr++;
}
}
Thank you.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] libbb: better coreutils compatibility for realpath
Applied, thanks.
On Sat, Jul 31, 2021 at 2:52 PM Ron Yorston wrote:
>
> Add some tests which coreutils realpath pass but BusyBox realpath
> fails (bar one). Adjust xmalloc_realpath_coreutils() so the tests
> pass:
>
> - Expand symbolic links before testing whether the last path component
> exists.
>
> - When the link target is a relative path canonicalize it by passing
> it through xmalloc_realpath_coreutils() as already happens for
> absolute paths.
>
> - Ignore trailing slashes when finding the last path component and
> correctly handle the case where the only slash is at the start of
> the path. This requires ignoring superfluous leading slashes.
>
> - Undo all changes to the path so error messages from the caller show
> the original filename.
>
> function old new delta
> xmalloc_realpath_coreutils 214 313 +99
>
> Signed-off-by: Ron Yorston
> ---
> include/libbb.h | 2 +-
> libbb/xreadlink.c| 75
> testsuite/realpath.tests | 45
> 3 files changed, 91 insertions(+), 31 deletions(-)
> create mode 100755 testsuite/realpath.tests
>
> diff --git a/include/libbb.h b/include/libbb.h
> index 7d6ab4a93..3c9ed792e 100644
> --- a/include/libbb.h
> +++ b/include/libbb.h
> @@ -561,7 +561,7 @@ DIR *xopendir(const char *path) FAST_FUNC;
> DIR *warn_opendir(const char *path) FAST_FUNC;
>
> char *xmalloc_realpath(const char *path) FAST_FUNC RETURNS_MALLOC;
> -char *xmalloc_realpath_coreutils(const char *path) FAST_FUNC RETURNS_MALLOC;
> +char *xmalloc_realpath_coreutils(char *path) FAST_FUNC RETURNS_MALLOC;
> char *xmalloc_readlink(const char *path) FAST_FUNC RETURNS_MALLOC;
> char *xmalloc_readlink_or_warn(const char *path) FAST_FUNC RETURNS_MALLOC;
> /* !RETURNS_MALLOC: it's a realloc-like function */
> diff --git a/libbb/xreadlink.c b/libbb/xreadlink.c
> index a18dd0748..2682f6975 100644
> --- a/libbb/xreadlink.c
> +++ b/libbb/xreadlink.c
> @@ -123,7 +123,7 @@ char* FAST_FUNC xmalloc_realpath(const char *path)
> #endif
> }
>
> -char* FAST_FUNC xmalloc_realpath_coreutils(const char *path)
> +char* FAST_FUNC xmalloc_realpath_coreutils(char *path)
> {
> char *buf;
>
> @@ -137,32 +137,19 @@ char* FAST_FUNC xmalloc_realpath_coreutils(const char
> *path)
> * (the directory must exist).
> */
> if (!buf && errno == ENOENT) {
> - char *last_slash = strrchr(path, '/');
> - if (last_slash) {
> - *last_slash++ = '\0';
> - buf = xmalloc_realpath(path);
> - if (buf) {
> - unsigned len = strlen(buf);
> - buf = xrealloc(buf, len + strlen(last_slash)
> + 2);
> - buf[len++] = '/';
> - strcpy(buf + len, last_slash);
> - }
> - } else {
> - char *target = xmalloc_readlink(path);
> - if (target) {
> - char *cwd;
> - if (target[0] == '/') {
> - /*
> -* $ ln -s /bin/qwe symlink # note:
> /bin is a link to /usr/bin
> -* $ readlink -f symlink
> -* /usr/bin/qwe/target_does_not_exist
> -* $ realpath symlink
> -* /usr/bin/qwe/target_does_not_exist
> -*/
> - buf =
> xmalloc_realpath_coreutils(target);
> - free(target);
> - return buf;
> - }
> + char *target, c, *last_slash;
> + size_t i;
> +
> + target = xmalloc_readlink(path);
> + if (target) {
> + /*
> +* $ ln -s /bin/qwe symlink # note: /bin is a link
> to /usr/bin
> +* $ readlink -f symlink
> +* /usr/bin/qwe
> +* $ realpath symlink
> +* /usr/bin/qwe
> +*/
> + if (target[0] != '/') {
> /*
> * $ ln -s target_does_not_exist symlink
> * $ readlink -f symlink
> @@ -170,13 +157,41 @@ char* FAST_FUNC xmalloc_realpath_coreutils(const char
> *path)
> * $ realpath symlink
> * /CURDIR/target_does_not_exist
> */
> - cwd =
Re: [PATCH v2 1/1] mktemp: add tmpdir option
Applied, thanks
On Fri, Jun 25, 2021 at 8:01 AM Andrej Valek wrote:
>
> Make mktemp more compatible with coreutils.
> - add "--tmpdir" option
> - add long variants for "d,q,u" options
>
> Note: Upstream ca-certificate update script started using this option.
>
> Signed-off-by: Andrej Valek
> Signed-off-by: Peter Marko
> ---
> coreutils/mktemp.c | 26 ++
> 1 file changed, 18 insertions(+), 8 deletions(-)
>
> diff --git a/coreutils/mktemp.c b/coreutils/mktemp.c
> index 5393320a5..05c6d98c6 100644
> --- a/coreutils/mktemp.c
> +++ b/coreutils/mktemp.c
> @@ -39,16 +39,17 @@
> //kbuild:lib-$(CONFIG_MKTEMP) += mktemp.o
>
> //usage:#define mktemp_trivial_usage
> -//usage: "[-dt] [-p DIR] [TEMPLATE]"
> +//usage: "[-dt] [-p DIR, --tmpdir[=DIR]] [TEMPLATE]"
> //usage:#define mktemp_full_usage "\n\n"
> //usage: "Create a temporary file with name based on TEMPLATE and
> print its name.\n"
> //usage: "TEMPLATE must end with XX (e.g. [/dir/]nameXX).\n"
> //usage: "Without TEMPLATE, -t tmp.XX is assumed.\n"
> -//usage: "\n -d Make directory, not file"
> -//usage: "\n -q Fail silently on errors"
> -//usage: "\n -t Prepend base directory name to TEMPLATE"
> -//usage: "\n -p DIR Use DIR as a base directory (implies -t)"
> -//usage: "\n -u Do not create anything; print a name"
> +//usage: "\n -d Make directory, not file"
> +//usage: "\n -q Fail silently on errors"
> +//usage: "\n -t Prepend base directory name
> to TEMPLATE"
> +//usage: "\n -p DIR, --tmpdir[=DIR] Use DIR as a base directory
> (implies -t)"
> +//usage: "\n For --tmpdir is a optional
> one."
> +//usage: "\n -u Do not create anything; print
> a name"
> //usage: "\n"
> //usage: "\nBase directory is: -p DIR, else $TMPDIR, else /tmp"
> //usage:
> @@ -72,13 +73,22 @@ int mktemp_main(int argc UNUSED_PARAM, char **argv)
> OPT_t = 1 << 2,
> OPT_p = 1 << 3,
> OPT_u = 1 << 4,
> + OPT_td = 1 << 5,
> };
>
> path = getenv("TMPDIR");
> if (!path || path[0] == '\0')
> path = "/tmp";
>
> - opts = getopt32(argv, "^" "dqtp:u" "\0" "?1"/*1 arg max*/, );
> + opts = getopt32long(argv, "^"
> + "dqtp:u\0"
> + "?1" /* 1 arg max */,
> + "directory\0" No_argument "d"
> + "quiet\0" No_argument "q"
> + "dry-run\0" No_argument "u"
> + "tmpdir\0"Optional_argument "\xff"
> + , ,
> + );
>
> chp = argv[optind];
> if (!chp) {
> @@ -95,7 +105,7 @@ int mktemp_main(int argc UNUSED_PARAM, char **argv)
> goto error;
> }
> #endif
> - if (opts & (OPT_t|OPT_p))
> + if (opts & (OPT_t|OPT_p|OPT_td))
> chp = concat_path_file(path, chp);
>
> if (opts & OPT_u) {
> --
> 2.11.0
>
> ___
> busybox mailing list
> busybox@busybox.net
> http://lists.busybox.net/mailman/listinfo/busybox
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] Fix const pointer assignment trick on clang 9+
On 08/10/2021 05:39, 余生与君 wrote:
And further investigation shows that a dummy function can also do this trick!
When one wants any sort of barrier (compiler optimization barrier, CPU
execution barrier, memory barriers, etc) that works (and is also
extremely likely to remain working in the future) for both GCC and
CLANG, it is usually a good idea to take a look at what the linux kernel
is using.
--8<-- from include/linux/compiler.h (latest linux kernel source)
/* Optimization barrier */
#ifndef barrier
/* The "volatile" is due to gcc bugs */
# define barrier() __asm__ __volatile__("": : :"memory")
#endif
#ifndef barrier_data
/*
* This version is i.e. to prevent dead stores elimination on @ptr
* where gcc and llvm may behave differently when otherwise using
* normal barrier(): while gcc behavior gets along with a normal
* barrier(), llvm needs an explicit input variable to be assumed
* clobbered. The issue is as follows: while the inline asm might
* access any memory it wants, the compiler could have fit all of
* @ptr into memory registers instead, and since @ptr never escaped
* from that, it proved that the inline asm wasn't touching any of
* it. This version works well with both compilers, i.e. we're telling
* the compiler that the inline asm absolutely may see the contents
* of @ptr. See also: https://llvm.org/bugs/show_bug.cgi?id=15495
*/
# define barrier_data(ptr) __asm__ __volatile__("": :"r"(ptr) :"memory")
#endif
--8<--
--
Henrique de Moraes Holschuh
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: devmem: use mmap64 replace mmap
On 08/Oct/2021 16:45, Denys Vlasenko wrote: > On Fri, Oct 8, 2021 at 10:16 AM xiechengliang > wrote: > > > > I have built busybox with CONFIG_LFS=y, this problem still exists. > > > > mmap is defined in glibc as below: > > void *mmap(void *addr, size_t len, int prot , int flags , int fd, off_t > > offset); > > The type of last parameter offset is off_ t, so using mmap will still cause > > truncation problems. > > > With CONFIG_LFS=y, off_t should be (at least) 64-bit. > > Tell me more about the architecture you are building for. > Is it bionic, maybe? They had some weird cases where off_t stays 32-bit. Looks like it could be bionic. At least none of: - glibc - uclibc - uclibc-ng - musl Targeting anything earlier than Android NDK API 21 probably will not work without workarounds. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: httpd ranges do not work over 4gb
On Fri, 8 Oct 2021 15:58:47 +0200 Denys Vlasenko wrote: > WRONG. If CONFIG_LFS is off, sizeof(off_t) can be larger than sizeof(int). > x86_64 is an example. > > > Fixed in git, please test. Fixed! Thanks. - Lauri ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Redirection of stdout for commands
Denys Vlasenko wrote: > [busybox] dd --help prints to stderr. D'oh! Lol, thanks! Any particular reason for this choice? It seems unintuitive to me to put normal output on an error stream, I guess separation of busybox itself and commands was wanted for some reason? On Fri, Oct 8, 2021 at 4:15 PM wrote: Hello there, 1st time post here. On a BusyBox based system (image built recently) I have a program calling external executables and reading their text output. This is done with fork, execl, and beforehand, redirecting the output with dup2, quite like here: https://dzone.com/articles/simple-popen2-implementation This generally works. There is one oddity, when calling Linux commands with this method: Depending on whether I run that same program on a regular distribution Linux machine, or on BusyBox, I get different results. It works as expected on e.g. Ubuntu. On BusyBox, it's different: - if I call only BusyBox with "busybox --help", rhings work as expected, e.g. nothing spills to the console, and I read all the input vie my redirected stdout busybox --help prints to stdout. - if I call a command handle by busybox, e.g. "dd --help", I get back nothing from stdout, and all text output spills directly on the console - if I call busybox explicitly, with the command added, like "busybox dd --help", it's the same as previous, i.e. doesn't work [busybox] dd --help prints to stderr. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: devmem: use mmap64 replace mmap
On Fri, Oct 8, 2021 at 10:16 AM xiechengliang wrote: > > I have built busybox with CONFIG_LFS=y, this problem still exists. > > mmap is defined in glibc as below: > void *mmap(void *addr, size_t len, int prot , int flags , int fd, off_t > offset); > The type of last parameter offset is off_ t, so using mmap will still cause > truncation problems. With CONFIG_LFS=y, off_t should be (at least) 64-bit. Tell me more about the architecture you are building for. Is it bionic, maybe? They had some weird cases where off_t stays 32-bit. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: Redirection of stdout for commands
On Fri, Oct 8, 2021 at 4:15 PM wrote: > Hello there, > > 1st time post here. > > On a BusyBox based system (image built recently) I have a program > calling external executables and reading their text output. > This is done with fork, execl, and beforehand, redirecting the output > with dup2, quite like here: > https://dzone.com/articles/simple-popen2-implementation > > This generally works. There is one oddity, when calling Linux commands > with this method: > > Depending on whether I run that same program on a regular distribution > Linux machine, or on BusyBox, I get different results. > It works as expected on e.g. Ubuntu. > On BusyBox, it's different: > - if I call only BusyBox with "busybox --help", rhings work as expected, > e.g. nothing spills to the console, and I read all the input vie my > redirected stdout busybox --help prints to stdout. > - if I call a command handle by busybox, e.g. "dd --help", I get back > nothing from stdout, and all text output spills directly on the console > - if I call busybox explicitly, with the command added, like "busybox dd > --help", it's the same as previous, i.e. doesn't work [busybox] dd --help prints to stderr. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] Fix const pointer assignment trick on clang 9+
On Fri, Oct 8, 2021 at 4:21 PM Denys Vlasenko wrote:
> Can you try this?
>
> #if defined(__clang_major__) && __clang_major__ >= 9
> /* Clang/llvm drops assignment to "constant" storage. Silently.
> * Needs serious convincing to not eliminate the store.
> */
> static ALWAYS_INLINE void register_barrier(void)
> {
> int r0, r1, r2, r3, r4, r5;
> asm volatile (
> "# clobber 6 registers"
> : "=r"(r0), "=r"(r1), "=r"(r2), "=r"(r3), "=r"(r4), "=r"(r5)
> : "0"(r1), "1"(r1), "2"(r2), "3"(r3), "4"(r4), "5"(r5)
> : "memory"
>
> );
> }
should be "0"(r0), not r1; also, dummy-initialize them to avoid warnings:
static ALWAYS_INLINE void register_barrier(void)
{
int r0 = r0, r1 = r1, r2 = r2, r3 = r3, r4 = r4, r5 = r5;
asm volatile (
"# clobber 6 registers"
: "=r"(r0), "=r"(r1), "=r"(r2), "=r"(r3), "=r"(r4), "=r"(r5)
: "0"(r0), "1"(r1), "2"(r2), "3"(r3), "4"(r4), "5"(r5)
: "memory"
);
}
6 registers is maximum what does not fail to compile for i386.
Size growth +1021 bytes.
5 registers: +564 bytes
4 registers: +424 bytes
3 registers: +303 bytes
2 registers: +205 bytes
1 register: +101 bytes
0 registers: 0 bytes (as expected)
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] Fix const pointer assignment trick on clang 9+
On Fri, Oct 8, 2021 at 10:39 AM 余生与君 wrote:
> On Thu, Oct 7, 2021 at 9:48 PM Denys Vlasenko
> wrote:
> > Can you try something more? E.g. (in current git):
> >
> > #define ASSIGN_CONST_PTR(p, v) do { \
> > *(void**)not_const_pp() = (void*)(v); \
> > /* At least gcc 3.4.6 on mipsel needs optimization barrier */ \
> > barrier(); \
> > + sleep(0); \
> > } while (0)
>
> Cool! Sleep(0) does the magic!
>
> ADRPX21, #ash_ptr_to_globals_misc_ptr@PAGE
> LDR X21, [X21,#ash_ptr_to_globals_misc_ptr@PAGEOFF]
> MOV X8, X21
> STR X0, [X8]
> MOV W0, WZR
> BL sleep
> LDR X21, [X21]
>
>
> And further investigation shows that a dummy function can also do this trick!
>
> hack.c:
> void clang_barrier() {
> }
We don't need the function call. We only need to tell clang that
a lot of registers are "needed" for something.
This "something" ideally should be nothing.
Can you try this?
#if defined(__clang_major__) && __clang_major__ >= 9
/* Clang/llvm drops assignment to "constant" storage. Silently.
* Needs serious convincing to not eliminate the store.
*/
static ALWAYS_INLINE void register_barrier(void)
{
int r0, r1, r2, r3, r4, r5;
asm volatile (
"# clobber 6 registers"
: "=r"(r0), "=r"(r1), "=r"(r2), "=r"(r3), "=r"(r4), "=r"(r5)
: "0"(r1), "1"(r1), "2"(r2), "3"(r3), "4"(r4), "5"(r5)
: "memory"
);
}
static ALWAYS_INLINE void* not_const_pp(const void *p)
{
void *pp;
asm volatile (
"# forget that p points to const"
: /*outputs*/ "=r" (pp)
: /*inputs*/ "0" (p)
);
return pp;
}
#define ASSIGN_CONST_PTR(p, v) do { \
*(void**)not_const_pp() = (void*)(v); \
register_barrier();
} while (0)
#else
static ALWAYS_INLINE void* not_const_pp(const void *p) { return (void*)p; }
#define ASSIGN_CONST_PTR(p, v) do { \
*(void**)not_const_pp() = (void*)(v); \
/* At least gcc 3.4.6 on mipsel needs optimization barrier */ \
barrier(); \
} while (0)
#endif
If this works, experiment what minimum number of rN
in register_barrier() is actually needed to placate clang.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Redirection of stdout for commands
Hello there, 1st time post here. On a BusyBox based system (image built recently) I have a program calling external executables and reading their text output. This is done with fork, execl, and beforehand, redirecting the output with dup2, quite like here: https://dzone.com/articles/simple-popen2-implementation This generally works. There is one oddity, when calling Linux commands with this method: Depending on whether I run that same program on a regular distribution Linux machine, or on BusyBox, I get different results. It works as expected on e.g. Ubuntu. On BusyBox, it's different: - if I call only BusyBox with "busybox --help", rhings work as expected, e.g. nothing spills to the console, and I read all the input vie my redirected stdout - if I call a command handle by busybox, e.g. "dd --help", I get back nothing from stdout, and all text output spills directly on the console - if I call busybox explicitly, with the command added, like "busybox dd --help", it's the same as previous, i.e. doesn't work Why is this, and can I make it work? Thanks, - Steve ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: httpd ranges do not work over 4gb
On Fri, Oct 8, 2021 at 2:45 PM Lauri Kasanen wrote:
>
> Hi,
>
> On current bb master httpd, when the client requests valid Range: bytes
> over 4gb, the response is wrong - it sends the entire file instead of
> the requested range.
>
> E.g.
> Range: bytes=5926192128-5926195199
>
> This is on x86_64, so CONFIG_LFS etc shouldn't matter. I haven't
> yet looked deeper if it's just a matter of using 64-bit vars.
Bug in libbb.h:
#else
/* CONFIG_LFS is off */
# if UINT_MAX == 0x
/* While sizeof(off_t) == sizeof(int), off_t is typedef'ed to long anyway.
* gcc will throw warnings on printf("%d", off_t). Crap... */
typedef unsigned long uoff_t;
# define XATOOFF(a) xatoi_positive(a)
# define BB_STRTOOFF bb_strtou
# define STRTOOFF strtol
# define OFF_FMT "l"
WRONG. If CONFIG_LFS is off, sizeof(off_t) can be larger than sizeof(int).
x86_64 is an example.
Fixed in git, please test.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
httpd ranges do not work over 4gb
Hi, On current bb master httpd, when the client requests valid Range: bytes over 4gb, the response is wrong - it sends the entire file instead of the requested range. E.g. Range: bytes=5926192128-5926195199 This is on x86_64, so CONFIG_LFS etc shouldn't matter. I haven't yet looked deeper if it's just a matter of using 64-bit vars. - Lauri bb-config.gz Description: Binary data ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: switch_root and /etc/inittab
I'm terribly late for this...
On Tue, Jun 15, 2021 at 4:55 PM Porter, Jeremy
wrote:
> Thank you for the example--that was helpful to see how this works.
>
> I have initramfs in Linux enabled and I've been using that for a long time. I
> should be able to use switch_root.
>
> I followed the prior suggestion to skip the inittab and use a script. I put a
> script in /etc/init.d/rcS:
>
> #!/bin/busybox sh
>
> /bin/busybox echo "Running boot script..."
> /bin/busybox echo "Mounting filesystems..."
> /bin/busybox mount -t devpts devpts /dev/pts
> /bin/busybox mount -t tmpfs tmpfs /dev/shm
> /bin/busybox mount -t proc proc /proc
> /bin/busybox mount -t sysfs sysfs /sys
> /bin/busybox mount -t tmpfs tmpfs /tmp
> /bin/busybox echo "Installing links for busybox..."
> /bin/busybox --install -s
> /bin/busybox echo /sbin/mdev > /proc/sys/kernel/hotplug
> /bin/busybox mdev -s
> /bin/busybox echo "mounting rootfs..."
> /bin/busybox mount -t xfs /dev/vda /mnt
> /bin/busybox echo "Swtiching root"
> exec /sbin/switch_root -c /dev/console /mnt /sbin/init
>
> This runs and gets to switch_root.
>
> It does not switch_root though. It doesn't run the new roots inittab which is
> present.
If switch_root can't/won't chroot to the new root, it prints an error message.
See for yourself:
// Change to new root directory and verify it's a different fs
xchdir(newroot);
xstat("/", );
rootdev = st.st_dev;
xstat(".", );
if (st.st_dev == rootdev) {
// Show usage, it says new root must be a mountpoint
bb_show_usage();
}
if (!dry_run && getpid() != 1) {
// Show usage, it says we must be PID 1
bb_show_usage();
}
// Additional sanity checks: we're about to rm -rf /, so be REALLY SURE
// we mean it. I could make this a CONFIG option, but I would get email
// from all the people who WILL destroy their filesystems.
if (stat("/init", ) != 0 || !S_ISREG(st.st_mode)) {
bb_error_msg_and_die("'%s' is not a regular file", "/init");
}
statfs("/", ); // this never fails
if ((unsigned)stfs.f_type != RAMFS_MAGIC
&& (unsigned)stfs.f_type != TMPFS_MAGIC
) {
bb_simple_error_msg_and_die("root filesystem is not
ramfs/tmpfs");
}
if (!dry_run) {
// Zap everything out of rootdev
delete_contents("/", rootdev);
// Overmount / with newdir and chroot into it
if (mount(".", "/", NULL, MS_MOVE, NULL)) {
// For example, fails when newroot is not a mountpoint
bb_simple_perror_msg_and_die("error moving root");
}
}
xchroot(".");
All error paths have messages. (x-functions (xstat et al) also always
print error messages if they die internally due to errors).
Thus, you should be reaching xchroot().
Then, you should be reaching execv, since it also has error printouts
everywhere else:
if (dry_run) {
// Does NEW_INIT look like it can be executed?
//xstat(argv[0], );
//if (!S_ISREG(st.st_mode))
// bb_perror_msg_and_die("'%s' is not a regular
file", argv[0]);
if (access(argv[0], X_OK) == 0)
return 0;
} else {
// Exec NEW_INIT
execv(argv[0], argv);
}
bb_perror_msg_and_die("can't execute '%s'", argv[0]);
> I get something like this:
>
> [1.262512] XFS (vda): Ending recovery (logdev: internal)
> Swtiching root
>
> Please press Enter to activate this console.
>
> BusyBox v1.33.1 (2021-06-10 08:53:34 EDT) built-in shell (ash)
>
> / # ls
Looks like you are seeing the new init running default
::askfirst:-/bin/sh
action.
Look around the currently mounted root filesystem and
diagnose why /etc/inittab is not there.
> Is there any good way to debug this switch_root?
You can strace it, I guess...
You can also use
exec /sbin/switch_root -c /dev/console /mnt /bin/sh
which would drop you into a shell, and then see around
what files are in the filesystem, try "exec /sbin/init" etc.
> The new root has /etc/inittab like this:
>
> ::sysinit:-/bin/ash
> ::sysinit:/bin/busybox echo "hello from /etc/inittab"
Do you *see* it in the filesystem when you are in the askfirst shell?
E.g. "cat /etc/inittab" shows the file?
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
[PATCH] Fix clang barrier for const trick by dummy func
From: YU Jincheng
- For clang 9+, `asm volatile ("":::"memory")`
does not ensure load after a const pointer
assigned by trick on `ASSIGN_CONST_PTR`
- We found that using a dummy function can do
this trick so this patch converts `barrier`
to an invisible dummy function
- This dummy function `barrier` should be
invisible from the caller to avoid inlining
by the compiler
- This trick introduces one instruction size
overhead which should only be 4-8 bytes and
the runtime overhead is at most one cache
miss (unconditional branch won't introduce
branch misprediction penalty)
Signed-off-by: LoveSy
---
include/libbb.h| 5 -
libbb/const_hack.c | 15 +++
2 files changed, 19 insertions(+), 1 deletion(-)
create mode 100644 libbb/const_hack.c
diff --git a/include/libbb.h b/include/libbb.h
index 02cc008f0..56b404b03 100644
--- a/include/libbb.h
+++ b/include/libbb.h
@@ -2284,7 +2284,6 @@ struct globals;
* If you want to assign a value, use SET_PTR_TO_GLOBALS(x) */
extern struct globals *const ptr_to_globals;
-#define barrier() asm volatile ("":::"memory")
#if defined(__clang_major__) && __clang_major__ >= 9
/* Clang/llvm drops assignment to "constant" storage. Silently.
@@ -2300,8 +2299,12 @@ static ALWAYS_INLINE void* not_const_pp(const void *p)
);
return pp;
}
+/* Clang/llvm needs an invisible dummy function for barrier
+ */
+void barrier();
#else
static ALWAYS_INLINE void* not_const_pp(const void *p) { return (void*)p; }
+#define barrier() asm volatile ("":::"memory")
#endif
#define ASSIGN_CONST_PTR(p, v) do { \
diff --git a/libbb/const_hack.c b/libbb/const_hack.c
new file mode 100644
index 0..5450b82c6
--- /dev/null
+++ b/libbb/const_hack.c
@@ -0,0 +1,15 @@
+/* vi: set sw=4 ts=4: */
+/*
+ * Dummy function as barrier for const trick
+ *
+ * Copyright (C) 2021 by YU Jincheng
+ *
+ * Licensed under GPLv2 or later, see file LICENSE in this source tree.
+ */
+
+#if defined(__clang_major__) && __clang_major__ >= 9
+void barrier() {
+ // dummy
+ // it should at most cost one instruction (4-8 bytes)
+}
+#endif
\ No newline at end of file
--
2.25.1
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: [PATCH] Fix const pointer assignment trick on clang 9+
On Thu, Oct 7, 2021 at 9:48 PM Denys Vlasenko wrote:
>
> On Thu, Oct 7, 2021 at 3:25 PM 余生与君 wrote:
> > > where p is a dummy, unused variable
> > No. p here shadows the global variable with the same name local
> > variable so the following context (scope) will use this local variable
> > instead of the global one.
>
> Aha...
> The problem here is that even though later uses of "p" in this block
> where we use ASSIGN_CONST_PTR() macro will use the local "p" pointer,
> when we exit the block, the following uses will refer to the global one.
> Nothing prevents them to still use incorrect value.
> The fix depends only on the hope that there won't be such uses.
> But they already exist:
>
> inetd.c:
> INIT_G();
> real_uid = getuid();
>
> ftpgetput.c:
> INIT_G();
> /* Set default values */
> user = "anonymous";
> password = "busybox";
Yes, I noticed that, and that's why I removed do-while in the INIT_G
in this patch.
> Can you try something more? E.g. (in current git):
>
> #define ASSIGN_CONST_PTR(p, v) do { \
> *(void**)not_const_pp() = (void*)(v); \
> /* At least gcc 3.4.6 on mipsel needs optimization barrier */ \
> barrier(); \
> + sleep(0); \
> } while (0)
Cool! Sleep(0) does the magic!
ADRPX21, #ash_ptr_to_globals_misc_ptr@PAGE
LDR X21, [X21,#ash_ptr_to_globals_misc_ptr@PAGEOFF]
MOV X8, X21
STR X0, [X8]
MOV W0, WZR
BL sleep
LDR X21, [X21]
And further investigation shows that a dummy function can also do this trick!
hack.c:
void clang_barrier() {
}
libbb.h:
void clang_barrier(); // invisible in this file
#define ASSIGN_CONST_PTR(p, v) do { \
*(void**)not_const_pp() = (void*)(v); \
/* At least gcc 3.4.6 on mipsel needs optimization barrier */ \
-barrier(); \
+ clang_barrier(); \
} while (0)
Result:
ADRPX21, #ash_ptr_to_globals_misc_ptr@PAGE
LDR X21, [X21,#ash_ptr_to_globals_misc_ptr@PAGEOFF]
MOV X8, X21
STR X0, [X8]
BL clang_barrier
LDR X21, [X21]
I will submit a patch about this.
> or even
>
> - barrier(); \
> + pselect(0, NULL, NULL, NULL, {0,0}, NULL); \
>
> Basically, we should convince clang to not load "p" too early,
> by starving it of available registers to store the fetched result.
>
> > > Also, why ({...;})
> > That is to avoid conflict between the global variable and the local
> > variable. When the ({...}) part is evaluating, it uses the global
> > variable p and stores it into the local variable, and then returns the
> > variable to initialize the local variable.
>
> I'm not even sure other compilers would interpret it like that -
> they might think that "p" inside ({}) is the newly defined one.
That's why I only apply it to clang to avoid other compilers having
different behavior.
But in c++14, this is UB. The `p` is the newly defined one. But we can
avoid this to two statements with a tmp variable. But the name can be
conflicted.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
答复: devmem: use mmap64 replace mmap
I have built busybox with CONFIG_LFS=y, this problem still exists.
mmap is defined in glibc as below:
void *mmap(void *addr, size_t len, int prot , int flags , int fd, off_t offset);
The type of last parameter offset is off_ t, so using mmap will still cause
truncation problems.
The mmap64() function is identical to the mmap() function except that it can be
used to map memory from files that are larger than 2 gigabytes into the process
memory.
Strace log :
arm32_for_busybox_devmem / # strace devmem 0x28000
execve("/sbin/devmem", ["devmem", "0x28000"], 0xbed2bd24 /* 21 vars */) = 0
...
...
...
mmap2(NULL, 4096, PROT_READ, MAP_SHARED, 3, 0x8000) = 0xb6f36000
...
...
...
exit_group(0) = ?
+++ exited with 0 +++
-邮件原件-
发件人: Denys Vlasenko [mailto:vda.li...@googlemail.com]
发送时间: 2021年10月7日 23:35
收件人: xiechengliang
抄送: busybox@busybox.net
主题: Re: devmem: use mmap64 replace mmap
This should be solvable by building with CONFIG_LFS=y
- this makes off_t 64-bit and makes all operations on files 64-bit aware.
Can you try that?
On Mon, Sep 27, 2021 at 4:06 PM xiechengliang wrote:
>
> Subject: [PATCH] devmem: The type of the physical address to be
> accessed by
>
> devmem is off_t, when the 32-bit Linux kernel support lpae, the
> physical
>
> address can exceed 4G, and accessing or modifying the address
> exceeding 4G
>
> will cause truncation.
>
>
>
> functionold new delta
>
> do_cmd127 127 +0
>
>
>
> Signed-off-by: xiechengliang
>
> ---
>
> miscutils/devmem.c | 6 +++---
>
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
>
>
> diff --git a/miscutils/devmem.c b/miscutils/devmem.c
>
> index f9f0276bc..594bb440c 100644
>
> --- a/miscutils/devmem.c
>
> +++ b/miscutils/devmem.c
>
> @@ -30,7 +30,7 @@ int devmem_main(int argc UNUSED_PARAM, char **argv)
>
>void *map_base, *virt_addr;
>
>uint64_t read_result;
>
>uint64_t writeval = writeval; /* for compiler */
>
> -off_t target;
>
> + uint64_t target;
>
>unsigned page_size, mapped_size, offset_in_page;
>
>int fd;
>
>unsigned width = 8 * sizeof(int);
>
> @@ -82,12 +82,12 @@ int devmem_main(int argc UNUSED_PARAM, char
> **argv)
>
> * Must map two pages to make it possible: */
>
> mapped_size *= 2;
>
>}
>
> -map_base = mmap(NULL,
>
> + map_base = mmap64(NULL,
>
> mapped_size,
>
> argv[3] ? (PROT_READ | PROT_WRITE) :
> PROT_READ,
>
> MAP_SHARED,
>
> fd,
>
> - target & ~(off_t)(page_size - 1));
>
> + target & ~(uint64_t)(page_size - 1));
>
>if (map_base == MAP_FAILED)
>
> bb_simple_perror_msg_and_die("mmap");
>
>
>
> --
>
> 2.12.3
>
>
>
> ___
> busybox mailing list
> busybox@busybox.net
> http://lists.busybox.net/mailman/listinfo/busybox
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
