Re: Stop using direct syscall(2) from perl(1)
On Thu, 3 Aug 2023 18:43:21 -0700
Andrew Hewus Fresh wrote:
> Here's a new version, I think I addressed all of the feedback I got,
> although may have missed something.
ok gkoehler@
You sent this new version 2 weeks ago, but I didn't build it until
yesterday. syscall_emulator.t passes on macppc and powerpc64.
> Index: gnu/usr.bin/perl/MANIFEST
> ===
> RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/MANIFEST,v
> retrieving revision 1.67
> diff -u -p -a -u -p -r1.67 MANIFEST
> --- gnu/usr.bin/perl/MANIFEST 8 Jul 2023 14:18:35 - 1.67
> +++ gnu/usr.bin/perl/MANIFEST 3 Aug 2023 04:34:38 -
> @@ -6605,6 +6605,7 @@ t/op/svleak.pl Test file for svleak.t
> t/op/svleak.tSee if stuff leaks SVs
> t/op/switch.tSee if switches (given/when) work
> t/op/symbolcache.t See if undef/delete works on stashes with
> functions
> +t/op/syscall_emulator.t Tests that syscall works via the
> emulator
> t/op/sysio.t See if sysread and syswrite work
> t/op/taint.t See if tainting works
> t/op/threads.t Misc. tests for perl features with
> threads
> Index: gnu/usr.bin/perl/Makefile.SH
> ===
> RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/Makefile.SH,v
> retrieving revision 1.60
> diff -u -p -a -u -p -r1.60 Makefile.SH
> --- gnu/usr.bin/perl/Makefile.SH 8 Jul 2023 14:18:35 - 1.60
> +++ gnu/usr.bin/perl/Makefile.SH 3 Aug 2023 04:34:38 -
> @@ -541,7 +541,7 @@ c1 = av.c scope.c op.c doop.c doio.c dum
> c2 = perly.c pp.c pp_hot.c pp_ctl.c pp_sys.c regcomp.c regexec.c utf8.c sv.c
> c3 = taint.c toke.c util.c deb.c run.c builtin.c universal.c pad.c globals.c
> keywords.c
> c4 = perlio.c numeric.c mathoms.c locale.c pp_pack.c pp_sort.c caretx.c
> dquote.c time64.c
> -c5 = $(mallocsrc)
> +c5 = $(mallocsrc) syscall_emulator.c
>
> !NO!SUBS!
>
> @@ -557,7 +557,7 @@ c = $(c1) $(c2) $(c3) $(c4) $(c5) minipe
>
> obj1 = $(mallocobj) gv$(OBJ_EXT) toke$(OBJ_EXT) perly$(OBJ_EXT)
> pad$(OBJ_EXT) regcomp$(OBJ_EXT) dump$(OBJ_EXT) util$(OBJ_EXT) mg$(OBJ_EXT)
> reentr$(OBJ_EXT) mro_core$(OBJ_EXT) keywords$(OBJ_EXT) builtin$(OBJ_EXT)
> obj2 = hv$(OBJ_EXT) av$(OBJ_EXT) run$(OBJ_EXT) pp_hot$(OBJ_EXT) sv$(OBJ_EXT)
> pp$(OBJ_EXT) scope$(OBJ_EXT) pp_ctl$(OBJ_EXT) pp_sys$(OBJ_EXT)
> -obj3 = doop$(OBJ_EXT) doio$(OBJ_EXT) regexec$(OBJ_EXT) utf8$(OBJ_EXT)
> taint$(OBJ_EXT) deb$(OBJ_EXT) globals$(OBJ_EXT) perlio$(OBJ_EXT)
> numeric$(OBJ_EXT) mathoms$(OBJ_EXT) locale$(OBJ_EXT) pp_pack$(OBJ_EXT)
> pp_sort$(OBJ_EXT) caretx$(OBJ_EXT) dquote$(OBJ_EXT) time64$(OBJ_EXT)
> +obj3 = doop$(OBJ_EXT) doio$(OBJ_EXT) regexec$(OBJ_EXT) utf8$(OBJ_EXT)
> taint$(OBJ_EXT) deb$(OBJ_EXT) globals$(OBJ_EXT) perlio$(OBJ_EXT)
> numeric$(OBJ_EXT) mathoms$(OBJ_EXT) locale$(OBJ_EXT) pp_pack$(OBJ_EXT)
> pp_sort$(OBJ_EXT) caretx$(OBJ_EXT) dquote$(OBJ_EXT) time64$(OBJ_EXT)
> syscall_emulator$(OBJ_EXT)
>
> # split the objects into 3 exclusive sets: those used by both miniperl and
> # perl, and those used by just one or the other. Doesn't include the
> Index: gnu/usr.bin/perl/Makefile.bsd-wrapper
> ===
> RCS file:
> /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/Makefile.bsd-wrapper,v
> retrieving revision 1.113
> diff -u -p -a -u -p -r1.113 Makefile.bsd-wrapper
> --- gnu/usr.bin/perl/Makefile.bsd-wrapper 15 Feb 2023 01:38:20 -
> 1.113
> +++ gnu/usr.bin/perl/Makefile.bsd-wrapper 3 Aug 2023 04:34:38 -
> @@ -39,11 +39,18 @@ cleandir:
> fi
> cd ${.CURDIR} && ${MAKE} -f Makefile.bsd-wrapper1 cleandir
>
> -all: config.sh
> +all: syscall_emulator.c config.sh
> cd ${.CURDIR} && exec ${MAKE} -f Makefile.bsd-wrapper1 perl.build
> cd ${.CURDIR} && exec ${MAKE} -f Makefile.bsd-wrapper1 mansrc.build
>
> install:
> cd ${.CURDIR} && exec ${MAKE} -f Makefile.bsd-wrapper1 install
> +
> +
> +syscall_emulator.c: gen_syscall_emulator.pl syscall_emulator.h
> /usr/include/sys/syscall.h /usr/include/sys/syscallargs.h
> + /usr/bin/perl $(.CURDIR)/gen_syscall_emulator.pl > $@
> +
> +syscall_emulator.h:
> + ln -sf $(.CURDIR)/$@ $@
>
> .include
> Index: gnu/usr.bin/perl/config.over
> ===
> RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/config.over,v
> retrieving revision 1.22
> diff -u -p -a -u -p -r1.22 config.over
> --- gnu/usr.bin/perl/config.over 5 Feb 2017 00:33:38 - 1.22
> +++ gnu/usr.bin/perl/config.over 3 Aug 2023 04:34:38 -
> @@ -64,3 +64,9 @@ myuname='openbsd'
>
> # force to use ranlib
> ranlib='ranlib'
> +
> +# Enable the syscall emulator,
> +# enabling syscall even if we don't have it
>
Re: Stop using direct syscall(2) from perl(1)
On Sun, Jul 09, 2023 at 01:29:58PM -0700, Andrew Hewus Fresh wrote:
> Here is a patch to replace perl(1)'s use of syscall(2) with a dispatcher
> that will call the libc function instead.
>
> I have to do some work on style before this is ready to commit, but it
> should be ready for some testing.
>
> I don't currently plan on committing syscall_emulator.c because we need
> to regenerate it each time as I'm not sure how to tie it into sys/kern's
> `make syscalls` to only do it when things change.
>
> Looking for tests from folks who use syscall from perl as well as style
> suggestions (like how do I correctly sort headers?) and maybe even
> ways to enable additional syscalls.
Here's a new version, I think I addressed all of the feedback I got,
although may have missed something.
Hopefully it is easier to test with sthen@'s fix.
Thanks miod@, espie@, sthen@, and especially gkoehler@ for a lot of
excellent feedback.
I'm sure there's still stuff missing and obvious problems that don't
stand out to someone who writes a lot of perl.
Starting Saturday I'm traveling for work for the week, so will probably
be even less responsive than usual until I'm back.
Index: gnu/usr.bin/perl/MANIFEST
===
RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/MANIFEST,v
retrieving revision 1.67
diff -u -p -a -u -p -r1.67 MANIFEST
--- gnu/usr.bin/perl/MANIFEST 8 Jul 2023 14:18:35 - 1.67
+++ gnu/usr.bin/perl/MANIFEST 3 Aug 2023 04:34:38 -
@@ -6605,6 +6605,7 @@ t/op/svleak.plTest file for svleak.t
t/op/svleak.t See if stuff leaks SVs
t/op/switch.t See if switches (given/when) work
t/op/symbolcache.t See if undef/delete works on stashes with
functions
+t/op/syscall_emulator.tTests that syscall works via the
emulator
t/op/sysio.t See if sysread and syswrite work
t/op/taint.t See if tainting works
t/op/threads.t Misc. tests for perl features with threads
Index: gnu/usr.bin/perl/Makefile.SH
===
RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/Makefile.SH,v
retrieving revision 1.60
diff -u -p -a -u -p -r1.60 Makefile.SH
--- gnu/usr.bin/perl/Makefile.SH8 Jul 2023 14:18:35 - 1.60
+++ gnu/usr.bin/perl/Makefile.SH3 Aug 2023 04:34:38 -
@@ -541,7 +541,7 @@ c1 = av.c scope.c op.c doop.c doio.c dum
c2 = perly.c pp.c pp_hot.c pp_ctl.c pp_sys.c regcomp.c regexec.c utf8.c sv.c
c3 = taint.c toke.c util.c deb.c run.c builtin.c universal.c pad.c globals.c
keywords.c
c4 = perlio.c numeric.c mathoms.c locale.c pp_pack.c pp_sort.c caretx.c
dquote.c time64.c
-c5 = $(mallocsrc)
+c5 = $(mallocsrc) syscall_emulator.c
!NO!SUBS!
@@ -557,7 +557,7 @@ c = $(c1) $(c2) $(c3) $(c4) $(c5) minipe
obj1 = $(mallocobj) gv$(OBJ_EXT) toke$(OBJ_EXT) perly$(OBJ_EXT) pad$(OBJ_EXT)
regcomp$(OBJ_EXT) dump$(OBJ_EXT) util$(OBJ_EXT) mg$(OBJ_EXT) reentr$(OBJ_EXT)
mro_core$(OBJ_EXT) keywords$(OBJ_EXT) builtin$(OBJ_EXT)
obj2 = hv$(OBJ_EXT) av$(OBJ_EXT) run$(OBJ_EXT) pp_hot$(OBJ_EXT) sv$(OBJ_EXT)
pp$(OBJ_EXT) scope$(OBJ_EXT) pp_ctl$(OBJ_EXT) pp_sys$(OBJ_EXT)
-obj3 = doop$(OBJ_EXT) doio$(OBJ_EXT) regexec$(OBJ_EXT) utf8$(OBJ_EXT)
taint$(OBJ_EXT) deb$(OBJ_EXT) globals$(OBJ_EXT) perlio$(OBJ_EXT)
numeric$(OBJ_EXT) mathoms$(OBJ_EXT) locale$(OBJ_EXT) pp_pack$(OBJ_EXT)
pp_sort$(OBJ_EXT) caretx$(OBJ_EXT) dquote$(OBJ_EXT) time64$(OBJ_EXT)
+obj3 = doop$(OBJ_EXT) doio$(OBJ_EXT) regexec$(OBJ_EXT) utf8$(OBJ_EXT)
taint$(OBJ_EXT) deb$(OBJ_EXT) globals$(OBJ_EXT) perlio$(OBJ_EXT)
numeric$(OBJ_EXT) mathoms$(OBJ_EXT) locale$(OBJ_EXT) pp_pack$(OBJ_EXT)
pp_sort$(OBJ_EXT) caretx$(OBJ_EXT) dquote$(OBJ_EXT) time64$(OBJ_EXT)
syscall_emulator$(OBJ_EXT)
# split the objects into 3 exclusive sets: those used by both miniperl and
# perl, and those used by just one or the other. Doesn't include the
Index: gnu/usr.bin/perl/Makefile.bsd-wrapper
===
RCS file:
/home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/Makefile.bsd-wrapper,v
retrieving revision 1.113
diff -u -p -a -u -p -r1.113 Makefile.bsd-wrapper
--- gnu/usr.bin/perl/Makefile.bsd-wrapper 15 Feb 2023 01:38:20 -
1.113
+++ gnu/usr.bin/perl/Makefile.bsd-wrapper 3 Aug 2023 04:34:38 -
@@ -39,11 +39,18 @@ cleandir:
fi
cd ${.CURDIR} && ${MAKE} -f Makefile.bsd-wrapper1 cleandir
-all: config.sh
+all: syscall_emulator.c config.sh
cd ${.CURDIR} && exec ${MAKE} -f Makefile.bsd-wrapper1 perl.build
cd ${.CURDIR} && exec ${MAKE} -f Makefile.bsd-wrapper1 mansrc.build
install:
cd ${.CURDIR} && exec ${MAKE} -f Makefile.bsd-wrapper1 install
+
+
+syscall_emulator.c: gen_syscall_emulator.pl syscall_emulator.h
/usr/include/sys/syscall.h
Re: Stop using direct syscall(2) from perl(1)
On Thu, 20 Jul 2023 19:32:33 -0700 Andrew Hewus Fresh wrote: > On Mon, Jul 17, 2023 at 11:54:18PM -0400, George Koehler wrote: > > How to pread($fd, $buf, 4, 16)? > > > > 1. syscall(169, $fd, $buf, 4, 0, 16) > > 2. syscall(169, $fd, $buf, 4, 0, 0, 16) > > 3. syscall(169, $fd, $buf, 4, 16, 0) > > 4. syscall(169, $fd, $buf, 4, 0, 16, 0) > > 5. syscall(169, $fd, $buf, 4, 16) > > Which is the way it works without the this middleman? This is on macppc. Without your diff is syscall(2). With your diff is va_args(args, off_t). pread(2) without your diff needs line 2, $ perl 3 -E 'syscall(169, 3, $_ = "c" x 7, 7, 0, 0, 11); say $_' Charlie pread(2) with your diff needs line 1, $ perl 3 -E 'syscall(169, 3, $_ = "c" x 7, 7, 0, 11); say $_' Charlie I don't know how it worked before we removed __syscall(2). > > $ cd /usr/src/gnu/usr.bin/perl/obj > > $ MALLOC_OPTIONS=S perl t/op/syscall_emulator.t > > /usr/include/sys/syscall.h -> /usr/include/sys/syscall.ph > > 1..13 > > ok 1 - Opened test.txt for write/create > > ok 2 - Wrote out to test.txt > > ok 3 - closed test.txt > > perl(15511) in malloc(): write after free 0xc36d1efcddc0 > > Abort trap (core dumped) > > Hmm, I guess I need to get a powerpc64 machine. You didn't need a powerpc64. The crash was random; if you ran the test multiple times, then it would crash. Your updated diff from Thu 20 Jul doesn't crash.
Re: Stop using direct syscall(2) from perl(1)
On Thu, Jul 20, 2023 at 10:59 PM Theo de Raadt wrote:
> Andrew Hewus Fresh wrote:
>
> > One thing to note, on the advice of miod@, I adjusted the mapping of
> > size_t from u_int to u_long, but that means we no longer recognize
> > getlogin_r. When I asked, miod@ suggested that syscalls.master was
> > confused.
> >
> > $ grep getlogin_r /usr/src/sys/kern/syscalls.master /usr/include/unistd.h
> > /usr/src/sys/kern/syscalls.master:141 STD { int
> sys_getlogin_r(char *namebuf, u_int namelen); }
> > /usr/include/unistd.h:intgetlogin_r(char *, size_t)
>
> That's quite a surprise.
>
> syscalls.master will need to be changed, which will change
> struct sys_getlogin_r_args, which is used inside sys_getlogin_r() in
> kern_prot.c. Don't get fooled by the advisory /* */ block you see
> there, it is using the generated struct. But the offset to load the
> field isn't changing, and luckily the incorrect field is smaller than
> the correct field, so I *think* there is no major ABI crank needed.
>
Yeah, at least for syscall args a u_int and a size_t both get passed in a
register-sized space on all our platforms' ABIs, so sys_getlogin_r_args
doesn't actually change size or offsets.
(If it was off_t or long long vs an int or long-sized arg, that would be
different, of course, on 32bit archs, but this change is in the clear.)
It does mean the syscall is implicitly reducing the len argument mod 2^32
on ILP32 archs, so it may fail with ERANGE if passed a 4GB buffer when it
should succeed, but I don't think that merits any sort of syspatch or such.
Philip Guenther
Re: Stop using direct syscall(2) from perl(1)
Andrew Hewus Fresh wrote:
> One thing to note, on the advice of miod@, I adjusted the mapping of
> size_t from u_int to u_long, but that means we no longer recognize
> getlogin_r. When I asked, miod@ suggested that syscalls.master was
> confused.
>
> $ grep getlogin_r /usr/src/sys/kern/syscalls.master /usr/include/unistd.h
> /usr/src/sys/kern/syscalls.master:141 STD { int
> sys_getlogin_r(char *namebuf, u_int namelen); }
> /usr/include/unistd.h:intgetlogin_r(char *, size_t)
That's quite a surprise.
syscalls.master will need to be changed, which will change
struct sys_getlogin_r_args, which is used inside sys_getlogin_r() in
kern_prot.c. Don't get fooled by the advisory /* */ block you see
there, it is using the generated struct. But the offset to load the
field isn't changing, and luckily the incorrect field is smaller than
the correct field, so I *think* there is no major ABI crank needed.
Re: Stop using direct syscall(2) from perl(1)
Here's a new patch, mostly for sthen@'s obvious fix, although I have
added some of gkoehler@'s suggestions, although I have some questions
and comments inline.
I am looking at some of the other suggestions, but they showed up while
I was testing this set.
One thing to note, on the advice of miod@, I adjusted the mapping of
size_t from u_int to u_long, but that means we no longer recognize
getlogin_r. When I asked, miod@ suggested that syscalls.master was
confused.
$ grep getlogin_r /usr/src/sys/kern/syscalls.master /usr/include/unistd.h
/usr/src/sys/kern/syscalls.master:141 STD { int
sys_getlogin_r(char *namebuf, u_int namelen); }
/usr/include/unistd.h:intgetlogin_r(char *, size_t)
On Thu, Jul 13, 2023 at 07:38:22AM +0100, Stuart Henderson wrote:
> On 2023/07/13 00:57, George Koehler wrote:
> > On Sun, 9 Jul 2023 13:29:58 -0700
> > Andrew Hewus Fresh wrote:
> >
> > > Here is a patch to replace perl(1)'s use of syscall(2) with a dispatcher
> > > that will call the libc function instead.
> >
> > patch(1) didn't "chmod +x gen_syscall_emulator.pl", but I needed to
> > do so to get around this this error,
>
> There shouldn't be 'x' bits in files committed to the tree, so the
> Makefile should explicitly run this with perl.
Oops, fixed!
On Mon, Jul 17, 2023 at 11:54:18PM -0400, George Koehler wrote:
> I prefer braces like this,
>
> case SYS_truncate: {
> ...
> break;
> }
Easy enough, I didn't see that called out in style(9).
> In my opinion, all va_arg()s should look like
>
> off_t length = (off_t)va_arg(args, long);
>
> because perl passes every argument as long (from pp_syscall in
> pp_sys.c). I worry that va_arg(args, off_t) would act strangely
> on platforms with 32-bit long and 64-bit off_t. Only a few syscalls
> have off_t arguments, and these few calls are almost useless in Perl,
> so this might not affect real programs.
>
> How to pread($fd, $buf, 4, 16)?
>
> 1. syscall(169, $fd, $buf, 4, 0, 16)
> 2. syscall(169, $fd, $buf, 4, 0, 0, 16)
> 3. syscall(169, $fd, $buf, 4, 16, 0)
> 4. syscall(169, $fd, $buf, 4, 0, 16, 0)
> 5. syscall(169, $fd, $buf, 4, 16)
>
> If va_args(args, off_t) takes 2 longs, then pick line 1 if big-endian,
> or line 3 if little-endian. If it skips a 3rd long for alignment,
> then pick line 2 or 4. If it takes 1 long, then pick line 5.
>
> If we (off_t)va_args(args, long), then it always takes 1 long, so
> every platform picks line 5, but the offset must fit in a long.
Which is the way it works without the this middleman?
On Wed, Jul 19, 2023 at 03:58:25PM -0400, George Koehler wrote:
> On Sun, 9 Jul 2023 13:29:58 -0700
> Andrew Hewus Fresh wrote:
>
> > +syscall_emulator.h Emulator to dispatch syscalls to libc
>
> This got installed as
> /usr/libdata/perl5/powerpc64-openbsd/CORE/syscall_emulator.h
I was wondering about why there was a list of headers in the Makefile,
I went cross eyed while trying to track that down, I'm testing what I
expect will remove that now.
> ExtUtils::MakeMaker is using cc -DHAS_SYSCALL_EMULATOR to build my XS
> module; the define causes to #include .
Oh right, I forgot that I hacked that in as I tried to figure out
_where_ I should do that. I will tidy that up too.
> Can we hide this feature from XS? I wish that only pp_sys.c and
> syscall_emulator.c would #include , that
> syscall_emulator.h would not be installed, and that
> -DHAS_SYSCALL_EMULATOR would not appear in non-core modules.
I can just patch it in without the define. I may have to do a bit more
when syscall is actually removed, since Configure probes for it, but I
_think_ I found the right define to do that, but it always takes me too
long staring at this before it makes sense and I've been kept too busy
away from computers to do that these last couple weeks. Preliminary
testing tells me it probably works.
> Also, the new test has a heap overflow, because $sb is too small for
> a struct stat,
>
> $ cd /usr/src/gnu/usr.bin/perl/obj
> $ MALLOC_OPTIONS=S perl t/op/syscall_emulator.t
> /usr/include/sys/syscall.h -> /usr/include/sys/syscall.ph
> 1..13
> ok 1 - Opened test.txt for write/create
> ok 2 - Wrote out to test.txt
> ok 3 - closed test.txt
> perl(15511) in malloc(): write after free 0xc36d1efcddc0
> Abort trap (core dumped)
Hmm, I guess I need to get a powerpc64 machine.
> I prevented the overflow by making $sb too big,
>
> --- t/op/syscall_emulator.t.beforeThu Jul 13 00:39:10 2023
> +++ t/op/syscall_emulator.t Wed Jul 19 15:32:40 2023
> @@ -47,7 +47,7 @@
> my $out = "Hello World\n";
> my $in = "\0" x 32;
> my ($in_p, $in_v);
> -my $sb = "\0\0\0\0";
> +my $sb = "\0" x 4096;
> my $st_mode;
>
> my $perms = S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH;
I applied this too. Thanks!
On Wed, Jul 19, 2023 at 05:26:22PM -0600, Theo de Raadt wrote:
> -my $sb = "\0\0\0\0";
> +my $sb = "\0" x 4096;
>
> That's pretty terrible. Does this language
Re: Stop using direct syscall(2) from perl(1)
On Sun, Jul 09, 2023 at 01:29:58PM -0700, Andrew Hewus Fresh wrote:
> Here is a patch to replace perl(1)'s use of syscall(2) with a dispatcher
> that will call the libc function instead.
>
> I have to do some work on style before this is ready to commit, but it
> should be ready for some testing.
>
> I don't currently plan on committing syscall_emulator.c because we need
> to regenerate it each time as I'm not sure how to tie it into sys/kern's
> `make syscalls` to only do it when things change.
>
> Looking for tests from folks who use syscall from perl as well as style
> suggestions (like how do I correctly sort headers?) and maybe even
> ways to enable additional syscalls.
Nits in the perl part.
> Index: gnu/usr.bin/perl/gen_syscall_emulator.pl
> ===
> RCS file: gnu/usr.bin/perl/gen_syscall_emulator.pl
> diff -N gnu/usr.bin/perl/gen_syscall_emulator.pl
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ gnu/usr.bin/perl/gen_syscall_emulator.pl 9 Jul 2023 19:42:50 -
> @@ -0,0 +1,354 @@
> +#!/usr/bin/perl
> +#$OpenBSD$ #
> +use v5.36;
> +
> +# Copyright (c) 2023 Andrew Hewus Fresh
> +#
> +# Permission to use, copy, modify, and distribute this software for any
> +# purpose with or without fee is hereby granted, provided that the above
> +# copyright notice and this permission notice appear in all copies.
> +#
> +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> +
> +my $includes = '/usr/include';
> +
> +# See also /usr/src/sys/kern/syscalls.master
> +my %syscalls = parse_syscalls(
> +"$includes/sys/syscall.h",
> +"$includes/sys/syscallargs.h",
> +);
> +delete $syscalls{MAXSYSCALL}; # not an actual function
> +
> +# The ordered list of all the headers we need
> +my @headers = qw<
> + sys/syscall.h
> + stdarg.h
> + errno.h
> +
> + sys/socket.h
> + sys/event.h
> + sys/futex.h
> + sys/ioctl.h
> + sys/ktrace.h
> + sys/mman.h
> + sys/mount.h
> + sys/msg.h
> + sys/poll.h
> + sys/ptrace.h
> + sys/resource.h
> + sys/select.h
> + sys/sem.h
> + sys/shm.h
> + sys/stat.h
> + sys/sysctl.h
> + sys/time.h
> + sys/uio.h
> + sys/wait.h
> +
> + dirent.h
> + fcntl.h
> + sched.h
> + signal.h
> + stdlib.h
> + stdio.h
> + syslog.h
> + tib.h
> + time.h
> + unistd.h
> +>;
> +
> +foreach my $header (@headers) {
> + my $file = "$includes/$header";
> + open my $fh, '<', $file or die "Unable to open $file: $!";
you could just autodie as you don't ever do anything fancy when open fails.
> + my $content = do { local $/; readline $fh };
> + close $fh;
> +
> + # Look for matching syscalls in this header
not sure that comment brings anything, or maybe you want a function, as
you tend to get fairly long winded.
> + foreach my $name (sort keys %syscalls) {
> + my $s = $syscalls{$name};
> + my $func_sig = find_func_sig($content, $name, $s);
> +
> + if (ref $func_sig) {
> + die "Multiple defs for $name <$header> <$s->{header}>"
> + if $s->{header};
> + $s->{func} = $func_sig;
> + $s->{header} = $header;
> + } elsif ($func_sig) {
> + $s->{mismatched_sig} = "$func_sig <$header>";
> + }
> + }
> +}
> +
> +say "/*\n * Generated from gen_syscall_emulator.pl\n */";
> +say "#include <$_>" for @headers;
> +print <<"EOL";
> +
> +long
> +syscall_emulator(int syscall, ...)
> +{
> + long ret = 0;
> + va_list args;
> + va_start(args, syscall);
> +
> + switch(syscall) {
> +EOL
> +
> +foreach my $name (
> + sort { $syscalls{$a}{id} <=> $syscalls{$b}{id} } keys %syscalls
> +) {
> + my %s = %{ $syscalls{$name} };
I never put spaces in this kind of construct, as I find it fairly readable
by itself
> +
> + # Some syscalls we can't emulate, so we comment those out.
> + $s{skip} //= "Indirect syscalls not supported"
> + if !$s{argtypes} && ($s{args}[-1] || '') eq '...';
> + $s{skip} //= "Mismatched func: $s{mismatched_sig}"
> + if $s{mismatched_sig} and not $s{func};
> + $s{skip} //= "No signature found in headers"
> + unless $s{header};
> +
> + my $ret = $s{ret} eq 'void' ? '' : 'ret = ';
> + $ret .= '(long)' if $s{ret} eq 'void *';
> +
> + my (@args, @defines);
> + my $argname = '';
> + if ($s{argtypes}) {
> +
Re: Stop using direct syscall(2) from perl(1)
-my $sb = "\0\0\0\0"; +my $sb = "\0" x 4096; That's pretty terrible. Does this language not have types?
Re: Stop using direct syscall(2) from perl(1)
On Sun, 9 Jul 2023 13:29:58 -0700 Andrew Hewus Fresh wrote: > +syscall_emulator.h Emulator to dispatch syscalls to libc This got installed as /usr/libdata/perl5/powerpc64-openbsd/CORE/syscall_emulator.h ExtUtils::MakeMaker is using cc -DHAS_SYSCALL_EMULATOR to build my XS module; the define causes to #include . Can we hide this feature from XS? I wish that only pp_sys.c and syscall_emulator.c would #include , that syscall_emulator.h would not be installed, and that -DHAS_SYSCALL_EMULATOR would not appear in non-core modules. Also, the new test has a heap overflow, because $sb is too small for a struct stat, $ cd /usr/src/gnu/usr.bin/perl/obj $ MALLOC_OPTIONS=S perl t/op/syscall_emulator.t /usr/include/sys/syscall.h -> /usr/include/sys/syscall.ph 1..13 ok 1 - Opened test.txt for write/create ok 2 - Wrote out to test.txt ok 3 - closed test.txt perl(15511) in malloc(): write after free 0xc36d1efcddc0 Abort trap (core dumped) I prevented the overflow by making $sb too big, --- t/op/syscall_emulator.t.before Thu Jul 13 00:39:10 2023 +++ t/op/syscall_emulator.t Wed Jul 19 15:32:40 2023 @@ -47,7 +47,7 @@ my $out = "Hello World\n"; my $in = "\0" x 32; my ($in_p, $in_v); -my $sb = "\0\0\0\0"; +my $sb = "\0" x 4096; my $st_mode; my $perms = S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH;
Re: Stop using direct syscall(2) from perl(1)
On Sun, 9 Jul 2023 13:29:58 -0700
Andrew Hewus Fresh wrote:
> + case SYS_truncate:
> + {
> + const char * path = va_arg(args, const char *);
> + off_t length = va_arg(args, off_t);
> + ret = truncate(path, length);
> + }
> + break;
I prefer braces like this,
case SYS_truncate: {
...
break;
}
In my opinion, all va_arg()s should look like
off_t length = (off_t)va_arg(args, long);
because perl passes every argument as long (from pp_syscall in
pp_sys.c). I worry that va_arg(args, off_t) would act strangely
on platforms with 32-bit long and 64-bit off_t. Only a few syscalls
have off_t arguments, and these few calls are almost useless in Perl,
so this might not affect real programs.
How to pread($fd, $buf, 4, 16)?
1. syscall(169, $fd, $buf, 4, 0, 16)
2. syscall(169, $fd, $buf, 4, 0, 0, 16)
3. syscall(169, $fd, $buf, 4, 16, 0)
4. syscall(169, $fd, $buf, 4, 0, 16, 0)
5. syscall(169, $fd, $buf, 4, 16)
If va_args(args, off_t) takes 2 longs, then pick line 1 if big-endian,
or line 3 if little-endian. If it skips a 3rd long for alignment,
then pick line 2 or 4. If it takes 1 long, then pick line 5.
If we (off_t)va_args(args, long), then it always takes 1 long, so
every platform picks line 5, but the offset must fit in a long.
The syscalls with off_t are
void *mmap(void *, size_t, int, int, int, off_t);
void *mquery(void *, size_t, int, int, int, off_t);
off_t lseek(int, off_t, int);
int truncate(const char *, off_t);
int ftruncate(int, off_t);
ssize_t pread(int, void *, size_t, off_t);
ssize_t pwrite(int, const void *, size_t, off_t);
ssize_t preadv(int, const struct iovec *, int, off_t);
ssize_t pwritev(int, const struct iovec *, int, off_t);
syscall(SYS_lseek, @args) would truncate its return from off_t to
long, but that is fine, because everyone should use Perl's sysseek.
POSIX::2008 in CPAN looks like a better way to call pread.
Re: Stop using direct syscall(2) from perl(1)
On 2023/07/13 00:57, George Koehler wrote: > On Sun, 9 Jul 2023 13:29:58 -0700 > Andrew Hewus Fresh wrote: > > > Here is a patch to replace perl(1)'s use of syscall(2) with a dispatcher > > that will call the libc function instead. > > patch(1) didn't "chmod +x gen_syscall_emulator.pl", but I needed to > do so to get around this this error, There shouldn't be 'x' bits in files committed to the tree, so the Makefile should explicitly run this with perl. > $ make -f Makefile.bsd-wrapper > /usr/src/gnu/usr.bin/perl/gen_syscall_emulator.pl > syscall_emulator.c > /bin/sh: /usr/src/gnu/usr.bin/perl/gen_syscall_emulator.pl: cannot execute - > Permission denied > *** Error 126 in /usr/src/gnu/usr.bin/perl (Makefile.bsd-wrapper:51 > 'syscall_emulator.c') >
Re: Stop using direct syscall(2) from perl(1)
On Sun, 9 Jul 2023 13:29:58 -0700 Andrew Hewus Fresh wrote: > Here is a patch to replace perl(1)'s use of syscall(2) with a dispatcher > that will call the libc function instead. patch(1) didn't "chmod +x gen_syscall_emulator.pl", but I needed to do so to get around this this error, $ make -f Makefile.bsd-wrapper /usr/src/gnu/usr.bin/perl/gen_syscall_emulator.pl > syscall_emulator.c /bin/sh: /usr/src/gnu/usr.bin/perl/gen_syscall_emulator.pl: cannot execute - Permission denied *** Error 126 in /usr/src/gnu/usr.bin/perl (Makefile.bsd-wrapper:51 'syscall_emulator.c')
Stop using direct syscall(2) from perl(1)
Here is a patch to replace perl(1)'s use of syscall(2) with a dispatcher
that will call the libc function instead.
I have to do some work on style before this is ready to commit, but it
should be ready for some testing.
I don't currently plan on committing syscall_emulator.c because we need
to regenerate it each time as I'm not sure how to tie it into sys/kern's
`make syscalls` to only do it when things change.
Looking for tests from folks who use syscall from perl as well as style
suggestions (like how do I correctly sort headers?) and maybe even
ways to enable additional syscalls.
Index: gnu/usr.bin/perl/MANIFEST
===
RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/MANIFEST,v
retrieving revision 1.67
diff -u -p -a -u -p -r1.67 MANIFEST
--- gnu/usr.bin/perl/MANIFEST 8 Jul 2023 14:18:35 - 1.67
+++ gnu/usr.bin/perl/MANIFEST 9 Jul 2023 19:42:24 -
@@ -6151,6 +6151,8 @@ SECURITY.md Add Security Policy for
Gi
sv.c Scalar value code
sv.h Scalar value header
sv_inline.hPerl_newSV_type and required defs
+syscall_emulator.c Emulator to dispatch syscalls to libc
+syscall_emulator.h Emulator to dispatch syscalls to libc
t/base/cond.t See if conditionals work
t/base/if.tSee if if works
t/base/lex.t See if lexical items work
@@ -6605,6 +6607,7 @@ t/op/svleak.plTest file for svleak.t
t/op/svleak.t See if stuff leaks SVs
t/op/switch.t See if switches (given/when) work
t/op/symbolcache.t See if undef/delete works on stashes with
functions
+t/op/syscall_emulator.tTests that syscall works via the
emulator
t/op/sysio.t See if sysread and syswrite work
t/op/taint.t See if tainting works
t/op/threads.t Misc. tests for perl features with threads
Index: gnu/usr.bin/perl/Makefile.SH
===
RCS file: /home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/Makefile.SH,v
retrieving revision 1.60
diff -u -p -a -u -p -r1.60 Makefile.SH
--- gnu/usr.bin/perl/Makefile.SH8 Jul 2023 14:18:35 - 1.60
+++ gnu/usr.bin/perl/Makefile.SH9 Jul 2023 19:42:24 -
@@ -534,14 +534,14 @@ h2 = embed.h form.h gv.h handy.h hv.h hv
h3 = pad.h patchlevel.h perl.h perlapi.h perly.h pp.h proto.h regcomp.h
h4 = regexp.h scope.h sv.h unixish.h util.h iperlsys.h thread.h
h5 = utf8.h warnings.h mydtrace.h op_reg_common.h l1_char_class_tab.h
-h6 = charclass_invlists.h
+h6 = charclass_invlists.h syscall_emulator.h
h = $(h1) $(h2) $(h3) $(h4) $(h5) $(h6)
c1 = av.c scope.c op.c doop.c doio.c dump.c gv.c hv.c mg.c reentr.c mro_core.c
perl.c
c2 = perly.c pp.c pp_hot.c pp_ctl.c pp_sys.c regcomp.c regexec.c utf8.c sv.c
c3 = taint.c toke.c util.c deb.c run.c builtin.c universal.c pad.c globals.c
keywords.c
c4 = perlio.c numeric.c mathoms.c locale.c pp_pack.c pp_sort.c caretx.c
dquote.c time64.c
-c5 = $(mallocsrc)
+c5 = $(mallocsrc) syscall_emulator.c
!NO!SUBS!
@@ -557,7 +557,7 @@ c = $(c1) $(c2) $(c3) $(c4) $(c5) minipe
obj1 = $(mallocobj) gv$(OBJ_EXT) toke$(OBJ_EXT) perly$(OBJ_EXT) pad$(OBJ_EXT)
regcomp$(OBJ_EXT) dump$(OBJ_EXT) util$(OBJ_EXT) mg$(OBJ_EXT) reentr$(OBJ_EXT)
mro_core$(OBJ_EXT) keywords$(OBJ_EXT) builtin$(OBJ_EXT)
obj2 = hv$(OBJ_EXT) av$(OBJ_EXT) run$(OBJ_EXT) pp_hot$(OBJ_EXT) sv$(OBJ_EXT)
pp$(OBJ_EXT) scope$(OBJ_EXT) pp_ctl$(OBJ_EXT) pp_sys$(OBJ_EXT)
-obj3 = doop$(OBJ_EXT) doio$(OBJ_EXT) regexec$(OBJ_EXT) utf8$(OBJ_EXT)
taint$(OBJ_EXT) deb$(OBJ_EXT) globals$(OBJ_EXT) perlio$(OBJ_EXT)
numeric$(OBJ_EXT) mathoms$(OBJ_EXT) locale$(OBJ_EXT) pp_pack$(OBJ_EXT)
pp_sort$(OBJ_EXT) caretx$(OBJ_EXT) dquote$(OBJ_EXT) time64$(OBJ_EXT)
+obj3 = doop$(OBJ_EXT) doio$(OBJ_EXT) regexec$(OBJ_EXT) utf8$(OBJ_EXT)
taint$(OBJ_EXT) deb$(OBJ_EXT) globals$(OBJ_EXT) perlio$(OBJ_EXT)
numeric$(OBJ_EXT) mathoms$(OBJ_EXT) locale$(OBJ_EXT) pp_pack$(OBJ_EXT)
pp_sort$(OBJ_EXT) caretx$(OBJ_EXT) dquote$(OBJ_EXT) time64$(OBJ_EXT)
syscall_emulator$(OBJ_EXT)
# split the objects into 3 exclusive sets: those used by both miniperl and
# perl, and those used by just one or the other. Doesn't include the
Index: gnu/usr.bin/perl/Makefile.bsd-wrapper
===
RCS file:
/home/afresh1/OpenBSD-perl/OP/cvs/src/gnu/usr.bin/perl/Makefile.bsd-wrapper,v
retrieving revision 1.113
diff -u -p -a -u -p -r1.113 Makefile.bsd-wrapper
--- gnu/usr.bin/perl/Makefile.bsd-wrapper 15 Feb 2023 01:38:20 -
1.113
+++ gnu/usr.bin/perl/Makefile.bsd-wrapper 9 Jul 2023 19:42:24 -
@@ -39,11 +39,15 @@ cleandir:
fi
cd ${.CURDIR} && ${MAKE} -f Makefile.bsd-wrapper1 cleandir
-all: config.sh
