Re: [not quite an RFC] shared bytecode/optree
"BS" == Benjamin Stuhl [EMAIL PROTECTED] writes: BS 1. Bytecode can just be mmap'ed or read in, no playing BS around with relocations on loading or games with RVAs BS (which can't be used anyway, since variable RVAs vary based BS on what's been allocated or freed earlier). (What is an RVA?) And how does the actual runtime use a relocatable pointer? If it is an offset, then any access becomes an add. And depending upon the source of the pointer, it would either be a real address or an offset. Or if everything is a handle, then each access requires two fetches. And I don't see where you avoided the relocation. The handle table that would come in with the bytecode would need to be adjusted to reflect the real address. I vaguly can see a TIL that uses machine code linkage (real machine code jumps) that perhaps could use relative addressing as not needing relocation. But I'm not sure that all architectures support long enough relative jumps/calls. Doing the actual relocation should be quite fast. I believe that all current executables have to be relocated upon loading. Not to mention the calls to shared modules/dlls. chaim -- Chaim FrenkelNonlinear Knowledge, Inc. [EMAIL PROTECTED] +1-718-236-0183
Re: [not quite an RFC] shared bytecode/optree
On Wed, Oct 25, 2000 at 11:45:54AM -0400, Chaim Frenkel wrote: I vaguly can see a TIL that uses machine code linkage (real machine code jumps) that perhaps could use relative addressing as not needing relocation. But I'm not sure that all architectures support long enough relative jumps/calls. Specific example where you can't: on ARM, the branch instructions (B and BL) are PC relative, but only have a 24 bit offset field. The address space is (now) 32 bit, so there's parts you can't reach without either calculating addresses (in another register) and MOVing them to the PC, or loading the PC from a branch table in memory. Nicholas Clark
Re: [not quite an RFC] shared bytecode/optree
--- Chaim Frenkel [EMAIL PROTECTED] wrote: "BS" == Benjamin Stuhl [EMAIL PROTECTED] writes: BS 1. Bytecode can just be mmap'ed or read in, no playing BS around with relocations on loading or games with RVAs BS (which can't be used anyway, since variable RVAs vary based BS on what's been allocated or freed earlier). (What is an RVA?) relative virtual address And how does the actual runtime use a relocatable pointer? If it is an offset, then any access becomes an add. And depending upon the source of the pointer, it would either be a real address or an offset. Or if everything is a handle, then each access requires two fetches. And I don't see where you avoided the relocation. The handle table that would come in with the bytecode would need to be adjusted to reflect the real address. I vaguly can see a TIL that uses machine code linkage (real machine code jumps) that perhaps could use relative addressing as not needing relocation. But I'm not sure that all architectures support long enough relative jumps/calls. Doing the actual relocation should be quite fast. I believe that all current executables have to be relocated upon loading. Not to mention the calls to shared modules/dlls. chaim -- Chaim Frenkel Nonlinear Knowledge, Inc. [EMAIL PROTECTED] +1-718-236-0183 My primary goal (it may not have come accross strongly enough) in this proposal was sharing bytecode between threads even with an ithreadsish model (variables are thread-private, except when explicitly shared). This requires that the bytecode not contain direct pointers to variables, but rather references with at least one level of indirection. Avoiding fixups/relocations and allowing bytecode to be mmap()ed are additional potential benefits. But my first goal was to not have one copy of each subroutine in File::Spec::Functions for each thread I run. -- BKS __ Do You Yahoo!? Yahoo! Messenger - Talk while you surf! It's FREE. http://im.yahoo.com/
Re: [not quite an RFC] shared bytecode/optree
In message [EMAIL PROTECTED] Nicholas Clark [EMAIL PROTECTED] wrote: Specific example where you can't: on ARM, the branch instructions (B and BL) are PC relative, but only have a 24 bit offset field. The address space is (now) 32 bit, so there's parts you can't reach without either calculating addresses (in another register) and MOVing them to the PC, or loading the PC from a branch table in memory. That is actually a word offset of course, so it can actually reach up to 26 bits away in bytes. Still not the full 32 though. Of course that only becomes a problem if your program is big enough to exceed 26 bits of address space, which is pretty unlikely. That or if the program occupies seriously disjoint areas of address space. Tom -- Tom Hughes ([EMAIL PROTECTED]) http://www.compton.nu/ ...Don't believe in astrology. We Scorpios aren't taken in by such things.
Re: [not quite an RFC] shared bytecode/optree
On Wed, Oct 25, 2000 at 06:23:20PM +0100, Tom Hughes wrote:
In message [EMAIL PROTECTED]
Nicholas Clark [EMAIL PROTECTED] wrote:
Specific example where you can't:
on ARM, the branch instructions (B and BL) are PC relative, but only have
a 24 bit offset field. The address space is (now) 32 bit, so there's parts
you can't reach without either calculating addresses (in another register)
and MOVing them to the PC, or loading the PC from a branch table in memory.
That is actually a word offset of course, so it can actually reach
up to 26 bits away in bytes. Still not the full 32 though.
Good point
Of course that only becomes a problem if your program is big enough
to exceed 26 bits of address space, which is pretty unlikely. That
or if the program occupies seriously disjoint areas of address space.
Which is likely:
nick@Bagpuss [test]$ uname -a
Linux Bagpuss.unfortu.net 2.2.17-rmk1 #5 Mon Sep 18 19:03:46 BST 2000 armv4l unknown
nick@Bagpuss [test]$ cat mmap.c
#include unistd.h
#include sys/mman.h
#include stdio.h
#include sys/types.h
#include sys/stat.h
#include fcntl.h
int main () {
int motd = open ("/etc/motd", O_RDONLY);
void *mapped, *malloced, *big;
if (motd 0) {
perror ("Failed to open /etc/motd");
return 1;
}
mapped = mmap(NULL, 1024, PROT_EXEC | PROT_READ | PROT_WRITE , MAP_PRIVATE, motd, 0);
malloced = malloc (1024);
big = malloc (1024*1024*32);
printf ("mapped = %p malloced = %p big = %p main = %p\n", mapped, malloced, big,
main);
return 0;
}
nick@Bagpuss [test]$ ./mmap
mapped = 0x40015000 malloced = 0x2008670 big = 0x40105008 main = 0x200040c
likewise x86
[nick@babyhippo nick]$ ./mmap
mapped = 0x40013000 malloced = 0x80498d0 big = 0x40109008 main = 0x80484a0
[nick@babyhippo nick]$ uname -a
Linux babyhippo.com 2.2.12-20 #1 Mon Sep 27 10:40:35 EDT 1999 i686 unknown
mmap gives you memory from somewhere disjoint. And for some malloc()
implementations (glibc2.1 here, but I've compiled Doug Lea's malloc on
Solaris and HP UX) will call mmap for a large request.
(And at least one out of Solaris and HP UX also gives you pointers greater
than 0x8000 from mmap())
Particularly likely if we're considering mmap()ing bytecode in
Nicholas Clark
[not quite an RFC] shared bytecode/optree
Firstly, by "bytecode" I mean a .pmc and by "optree" I mean
the perl6 VM's internal form that it goes through
executing.
It seems to me that one thing that the perl6 bytecode
implementation _should_ do (in the interests of being light
and fast, as well as meshing well with MT) is be
position-independant. What do I mean? That all direct
references to SV*'s or regexes or anything else in the
bytecode _and_ the optree should actually be handles of
some sort. This has several benefits:
1. Bytecode can just be mmap'ed or read in, no playing
around with relocations on loading or games with RVAs
(which can't be used anyway, since variable RVAs vary based
on what's been allocated or freed earlier).
2. (more importantly, IMHO) Bytecode and the optree are
shareable between threads. My primary reason for opposing
to the RFC proposing that modules must be reloaded in each
thread is the immense amount of memory that would be wasted
without bytecode/optree sharing.
3. With a good slab allocator and possibly some mprotect()
calls (and a good OS) bytecode/optree suddenly becomes
_completely_ shared between child processed. No more
needing to restart httpd and mod_perl6 because the mixing
of code and data has doubled the core usage of each
process!
I don't have the background to seriously argue
implementation, but I might suggest a "handle table" of
sorts which defines for each thread and CV which variable
goes with which handle. This sort of ties in with my
(vague) idea that CVs should carry around instructions for
building their scratchpad, rather than the pad itself (IOW,
scratchpads become purely part of the stack frame, rather
than the subroutine's carrier variable). This is all for
the purpose of reducing the required locking around
subroutine calls to nil or almost nil (perhaps one to make
sure that no-one's changed the subroutine out from under us
via eval("*foo = \bar;"); or the like).
At any rate, I'm just spouting off ideas sparked by various
recent discussions (I probably need a higher blood sugar or
something). It's probably too early to seriously argue
technical merits, but on the other hand, basic VM design
can start before we know the precise grammar.
-- BKS
__
Do You Yahoo!?
Yahoo! Messenger - Talk while you surf! It's FREE.
http://im.yahoo.com/
Re: [not quite an RFC] shared bytecode/optree
On Tue, Oct 24, 2000 at 04:41:38PM -0700, Benjamin Stuhl wrote: It seems to me that one thing that the perl6 bytecode implementation _should_ do (in the interests of being light and fast, as well as meshing well with MT) is be position-independant. Fancy offering a patch to RFC310? -- If they can put a man on the moon, why can't they put them all there?
