Re: Bytecode metadata

2003-02-05 Thread James Michael DuPont
--- Leopold Toetsch [EMAIL PROTECTED] wrote: James Michael DuPont wrote: --- [EMAIL PROTECTED] wrote: Mike -- Thats a lot of metadata. OK that sounds fine. My current problems with the graphs of meta-data are the speed of loading. When you arrange the meta-data as a

Re: Bytecode metadata

2003-02-05 Thread Gopal V
If memory serves me right, James Michael DuPont wrote: JVM had a LineNumberTable and VarNameTable for debugging which were declared as ``attributes'' to each method in the .class tree. I suppose VarNameTable is totally irrelevant for Parrot ... I dont know that, what is it? Variable

Re: Bytecode metadata

2003-02-04 Thread James Michael DuPont
Dear All, I just wanted to ask about a conclusion on the bytecode metadata. Here are the things I would like to know about a given bytecode : what line (maybe column) it comes from Possible comments about it. If it is a method call, what is the method name,signature,locatoin If it is a variable

Re: Bytecode metadata

2003-02-04 Thread Leopold Toetsch
James Michael DuPont wrote: Dear All, I just wanted to ask about a conclusion on the bytecode metadata. Here are the things I would like to know about a given bytecode : what line (maybe column) it comes from File/line information is already there (imcc -d -o...) and working. If it is a

Re: Bytecode metadata

2003-02-04 Thread Juergen Boemmels
[EMAIL PROTECTED] writes: Mike -- Thats a lot of metadata. Sounds like maybe the metadata is primary and the bytecode is secondary, in which case perhaps what you really want is a (metadata) tree decorated with bytecode rather than a (bytecode) array decorated with metadata. The bytecode

Re: Bytecode metadata

2003-02-04 Thread gregor
PROTECTED] cc: Perl6 Internals [EMAIL PROTECTED] Subject:Re: Bytecode metadata [EMAIL PROTECTED] writes: Mike -- Thats a lot of metadata. Sounds like maybe the metadata is primary and the bytecode is secondary, in which case perhaps what you really want

Re: Bytecode metadata

2003-02-04 Thread James Michael DuPont
Juergen, I completly agree with you. For my needs, the meta-data does not have to be loaded at the same time at all. I can be in a different file for I care. I just want to know how where we can put it. The Microsoft IL has a whole section on meta-data, and one wonders what Parrot might be doing

Re: Bytecode metadata

2003-02-04 Thread Gopal V
If memory serves me right, James Michael DuPont wrote: I just want to know how where we can put it. The Microsoft IL has a whole section on meta-data, AFAIK, that just holds the offset, line number and filename. IIRC the JVM had a LineNumberTable and VarNameTable for debugging which were

Re: Bytecode metadata

2003-02-04 Thread James Michael DuPont
--- [EMAIL PROTECTED] wrote: Mike -- Thats a lot of metadata. Sounds like maybe the metadata is primary and the bytecode is secondary, in which case perhaps what you really want is a (metadata) tree decorated with bytecode rather than a (bytecode) array decorated with metadata. Fair

Re: Bytecode metadata

2003-02-04 Thread James Michael DuPont
Hey Gopal, Nice to meet you here ;) --- Gopal V [EMAIL PROTECTED] wrote: If memory serves me right, James Michael DuPont wrote: I just want to know how where we can put it. The Microsoft IL has a whole section on meta-data, AFAIK, that just holds the offset, line number and filename.

Re: Bytecode metadata

2003-02-04 Thread Leopold Toetsch
James Michael DuPont wrote: --- [EMAIL PROTECTED] wrote: Mike -- Thats a lot of metadata. OK that sounds fine. My current problems with the graphs of meta-data are the speed of loading. When you arrange the meta-data as a single opcode stream, you have ~zero load time for the mmap()ed

Re: Bytecode metadata

2003-01-28 Thread martin
On Sun, 26 Jan 2003, James Mastros wrote: just define a new packfile section, SIGNATURE, that is defined to be a cryptographic signature of all sections previous to it in the file. I'm battling with this in another file format at the moment; if possible can we please *not* have it sensitive to

Re: [Introspector-developers] Re: Bytecode metadata

2003-01-28 Thread Gopal V
If memory serves me right, James Michael DuPont wrote: Bah. That's parrot -o foo.o foo.pmc isn't it? And if we make C a parrot supported language we can even build parrot with parrot? Hmmm... bootstrapping 1. The gcc : I have %99 of the information about the function bodies of

Re: Bytecode metadata

2003-01-27 Thread Juergen Boemmels
Nicholas Clark [EMAIL PROTECTED] writes: [...] struct Chunk { opcode_t type; opcode_t version; opcode_t size; void data[]; }; will this ever compile? void data[] is not allowed, and even char data[] is an incomplete type, so

Re: Bytecode metadata

2003-01-27 Thread Andy Dougherty
On 27 Jan 2003, Juergen Boemmels wrote: Nicholas Clark [EMAIL PROTECTED] writes: struct Chunk { opcode_t type; opcode_t version; opcode_t size; void data[]; }; I agree with the roughly bit, but I'd suggest ensuring that you put

Re: Bytecode metadata

2003-01-27 Thread Leopold Toetsch
Juergen Boemmels wrote: Nicholas Clark [EMAIL PROTECTED] writes: struct Chunk { opcode_t type; opcode_t version; opcode_t size; void data[]; }; will this ever compile? It's similar to opcode_t *data. If size == 0, no data follow in byte stream. byte_code_{un,}pack is

Re: [Introspector-developers] Re: Bytecode metadata

2003-01-27 Thread James Michael DuPont
--- Juergen Boemmels [EMAIL PROTECTED] wrote: Nicholas Clark [EMAIL PROTECTED] writes: I guess in future once the normal JIT works, and we've got the pigs flying nicely then it would be possible to write a Not Just In Time compiler that saves out assembly code and relocation

Re: Bytecode metadata

2003-01-26 Thread Leopold Toetsch
Sean O'Rourke wrote: How true. On Solaris, for example, mmap's are aligned on 64k boundaries, which leads to horrible virtual address space consumption when you map lots of small things. If we're mmap()ing things, we want to be sure they're fairly large. Is one PBC file a small thing? Or in

Re: Bytecode metadata

2003-01-26 Thread Dave Mitchell
On Sat, Jan 25, 2003 at 05:38:08PM -0800, Sean O'Rourke wrote: The problem's actually _virtual_ memory use/fragmentation, not physical memory or swap. Say you map in 10k small files -- that's 640M virtual memory, just over a fourth of what's available. Now let's say you're also using mmap()

Re: Bytecode metadata

2003-01-26 Thread Sean O'Rourke
On Sat, 25 Jan 2003, Leopold Toetsch wrote: Is one PBC file a small thing? Or in other words, should we have a low limit where we start again to malloc and copy PBC files? Configure option? Commandline switch? Maybe a config option? The app I'm thinking of was pathological, in that it mapped

Re: Bytecode metadata

2003-01-26 Thread James Mastros
On 01/25/2003 4:26 AM, Leopold Toetsch wrote: Nicholas Clark wrote: Also some way of storing a cryptographic signature in the file, so that you could compile a parrot that automatically refuses to load code that isn't signed by you. The palladium parrot :) Just because it's possible to use a

Re: Bytecode metadata

2003-01-26 Thread James Mastros
On 01/26/2003 2:18 PM, Sean O'Rourke wrote: Maybe a config option? The app I'm thinking of was pathological, in that it mapped in thousands of 20-byte files. Now that I think about it, unless someone implements something very strangely (or has absolutely enormous numbers of threads) this

Re: Bytecode metadata

2003-01-25 Thread Leopold Toetsch
Nicholas Clark wrote: On Thu, Jan 23, 2003 at 02:48:38PM -0800, Brent Dax wrote: struct Chunk { opcode_t type; opcode_t version; opcode_t size; void data[]; }; I agree with the roughly bit, but I'd suggest ensuring that you put in enough bits to get data[] 64 bit aligned. If

Re: Bytecode metadata

2003-01-25 Thread Nicholas Clark
On Sat, Jan 25, 2003 at 10:26:22AM +0100, Leopold Toetsch wrote: Nicholas Clark wrote: Also some way of storing a cryptographic signature in the file, so that you could compile a parrot that automatically refuses to load code that isn't signed by you. The palladium parrot :) naa. I said

Re: Bytecode metadata

2003-01-25 Thread Leopold Toetsch
Dan Sugalski wrote: At 5:32 PM + 1/24/03, Dave Mitchell wrote: I just wrote a quick C program that successfully mmap-ed in all 1639 files in my Linux box's /usr/share/man/man1 directory. Linux is not the universe, though. I have it changed to use mmap() bytecode (other segments, with

Re: Bytecode metadata

2003-01-25 Thread Leopold Toetsch
Nicholas Clark wrote: On Sat, Jan 25, 2003 at 10:26:22AM +0100, Leopold Toetsch wrote: The palladium parrot :) naa. I said signed by you, not signed by the RIAA^WMPAA^WMicrosoft Yes, of course. I would do this with a personalized version of fingerprint.c and generate a separate

Re: Bytecode metadata

2003-01-25 Thread Sean O'Rourke
On Sat, 25 Jan 2003, Leopold Toetsch wrote: Dan Sugalski wrote: At 5:32 PM + 1/24/03, Dave Mitchell wrote: I just wrote a quick C program that successfully mmap-ed in all 1639 files in my Linux box's /usr/share/man/man1 directory. Linux is not the universe, though. How true.

Re: Bytecode metadata

2003-01-25 Thread Jason Gloudon
On Thu, Jan 23, 2003 at 08:39:21PM +, Dave Mitchell wrote: This means that a Perl server that relies on a lot of modules, and which forks for each connection (imagine a Perl-based web server), doesn't consume acres of swap space just to have an in-memory image per Perl process, of all the

Re: Bytecode metadata

2003-01-25 Thread Dave Mitchell
On Sat, Jan 25, 2003 at 06:18:47AM -0800, Sean O'Rourke wrote: On Sat, 25 Jan 2003, Leopold Toetsch wrote: Dan Sugalski wrote: At 5:32 PM + 1/24/03, Dave Mitchell wrote: I just wrote a quick C program that successfully mmap-ed in all 1639 files in my Linux box's

Re: Bytecode metadata

2003-01-25 Thread Dave Mitchell
On Sat, Jan 25, 2003 at 10:04:37AM -0500, Jason Gloudon wrote: On Thu, Jan 23, 2003 at 08:39:21PM +, Dave Mitchell wrote: This means that a Perl server that relies on a lot of modules, and which forks for each connection (imagine a Perl-based web server), doesn't consume acres of swap

Re: Bytecode metadata

2003-01-25 Thread Nicholas Clark
On Sat, Jan 25, 2003 at 11:43:40PM +, Dave Mitchell wrote: On Sat, Jan 25, 2003 at 06:18:47AM -0800, Sean O'Rourke wrote: On Sat, 25 Jan 2003, Leopold Toetsch wrote: Dan Sugalski wrote: At 5:32 PM + 1/24/03, Dave Mitchell wrote: I just wrote a quick C program that

Re: Bytecode metadata

2003-01-25 Thread Dave Mitchell
On Sun, Jan 26, 2003 at 12:40:19AM +, Nicholas Clark wrote: On Sat, Jan 25, 2003 at 11:43:40PM +, Dave Mitchell wrote: Okay, I just ran a program on a a Solaris machines that mmaps in each of 571 man files 20 times (a total of 11420 mmaps). The process size was 181Mb, but the total

Re: Bytecode metadata

2003-01-25 Thread Sean O'Rourke
On Sat, 25 Jan 2003, Dave Mitchell wrote: On Sat, Jan 25, 2003 at 06:18:47AM -0800, Sean O'Rourke wrote: On Sat, 25 Jan 2003, Leopold Toetsch wrote: Dan Sugalski wrote: At 5:32 PM + 1/24/03, Dave Mitchell wrote: I just wrote a quick C program that successfully mmap-ed in

Re: Bytecode metadata

2003-01-24 Thread Leopold Toetsch
Juergen Boemmels wrote: Dan Sugalski [EMAIL PROTECTED] writes: It might be even possible to dump the jitted code. This would increase the startup. Then strip the bytecode to reduce the size of the file and TADA: Yet another new binary format. When you then are able to to get the same

Re: Bytecode metadata

2003-01-24 Thread Leopold Toetsch
Dan Sugalski wrote: At 8:39 PM + 1/23/03, Dave Mitchell wrote: in such a way that it (or most of it) can simply be mmap-ed in (RO), analogously to executables. This is the way the bytecode currently works, and we will *not* switch to any bytecode format that doesn't at least allow the

Re: Bytecode metadata

2003-01-24 Thread Leopold Toetsch
Leopold Toetsch wrote: I'm currently simplifying the whole packfile routines. It still does read the old format, but the compat code is centralized now in one place. Registered types are consecutively numbered, unknown types still get unpacked or dumped: typedef enum { PF_DIR_SEG,

Re: Bytecode metadata

2003-01-24 Thread Dave Mitchell
On Fri, Jan 24, 2003 at 07:23:04AM +0100, Leopold Toetsch wrote: How many mmap's can $arch have for one program and for all? Could we hit some limits here, if every module loaded gets (and stays) mmap()ed. I just wrote a quick C program that successfully mmap-ed in all 1639 files in my Linux

Re: Bytecode metadata

2003-01-24 Thread Dan Sugalski
At 5:32 PM + 1/24/03, Dave Mitchell wrote: On Fri, Jan 24, 2003 at 07:23:04AM +0100, Leopold Toetsch wrote: How many mmap's can $arch have for one program and for all? Could we hit some limits here, if every module loaded gets (and stays) mmap()ed. I just wrote a quick C program that

Re: Bytecode metadata

2003-01-24 Thread Nicholas Clark
On Thu, Jan 23, 2003 at 02:48:38PM -0800, Brent Dax wrote: Are you expecting to have chunk type determined by order? If so, what will you do if a future restructuring means you either don't need chunk type X or you need a new, highly incompatible version? Will you leave in an empty ghost

RE: Bytecode metadata

2003-01-23 Thread Brent Dax
Dan Sugalski: # Since it looks like it's time to extend the packfile format and the # in-memory bytecode layout, this would be the time to start discussing # metadata. What sorts of metadata do people think are useful to have # in either the packfile (on disk) or in the bytecode (in memory). I

Re: Bytecode metadata

2003-01-23 Thread chromatic
On Wed, 22 Jan 2003 13:27:47 +, Dan Sugalski wrote: Since it looks like it's time to extend the packfile format and the in-memory bytecode layout, this would be the time to start discussing metadata. What sorts of metadata do people think are useful to have in either the packfile (on

Re: Bytecode metadata

2003-01-23 Thread Juergen Boemmels
Hello, after quite a long time away from keyboard and fighting through a huge backlog of mail I'm (hopefully) back again. Dan Sugalski [EMAIL PROTECTED] writes: Since it looks like it's time to extend the packfile format and the in-memory bytecode layout, this would be the time to start

Re: Bytecode metadata

2003-01-23 Thread Dave Mitchell
On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged in such a way that it (or most of it) can simply be mmap-ed in (RO), analogously to executables.

Re: Bytecode metadata

2003-01-23 Thread James Michael DuPont
--- chromatic [EMAIL PROTECTED] wrote: On Wed, 22 Jan 2003 13:27:47 +, Dan Sugalski wrote: Since it looks like it's time to extend the packfile format and the in-memory bytecode layout, this would be the time to start discussing metadata. What sorts of metadata do people think

Re: Bytecode metadata

2003-01-23 Thread James Michael DuPont
--- Dave Mitchell [EMAIL PROTECTED] wrote: On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged in such a way that it (or most of it) can simply

Re: Bytecode metadata

2003-01-23 Thread Juergen Boemmels
Dave Mitchell [EMAIL PROTECTED] writes: On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged in such a way that it (or most of it) can simply be

Re: Bytecode metadata

2003-01-23 Thread Dan Sugalski
At 10:31 PM +0100 1/23/03, Juergen Boemmels wrote: Dave Mitchell [EMAIL PROTECTED] writes: On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged

Re: Bytecode metadata

2003-01-23 Thread Juergen Boemmels
Dan Sugalski [EMAIL PROTECTED] writes: This might be possible if the byteorder, wordsize, defaultencoding etc. are the same in the file on disk and the host. Which will generally be the case, I expect. Tell a sysadmin that they can reduce the memory footprint of mod_parrot by 50% by running

RE: Bytecode metadata

2003-01-23 Thread Brent Dax
Dan Sugalski: # At 12:10 AM -0800 1/23/03, Brent Dax wrote: # Dan Sugalski: # # Since it looks like it's time to extend the packfile # format and the # # in-memory bytecode layout, this would be the time to start # discussing # # metadata. What sorts of metadata do people think are useful #

Re: Bytecode metadata

2003-01-23 Thread Dan Sugalski
At 8:39 PM + 1/23/03, Dave Mitchell wrote: On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged in such a way that it (or most of it) can simply

Re: Bytecode metadata

2003-01-23 Thread Dan Sugalski
At 11:48 AM -0800 1/23/03, chromatic wrote: On Wed, 22 Jan 2003 13:27:47 +, Dan Sugalski wrote: Since it looks like it's time to extend the packfile format and the in-memory bytecode layout, this would be the time to start discussing metadata. What sorts of metadata do people think are

Re: Bytecode metadata

2003-01-23 Thread James Michael DuPont
--- Juergen Boemmels [EMAIL PROTECTED] wrote: Hello, after quite a long time away from keyboard and fighting through a huge backlog of mail I'm (hopefully) back again. Dan Sugalski [EMAIL PROTECTED] writes: Since it looks like it's time to extend the packfile format and the

RE: Bytecode metadata

2003-01-23 Thread Dan Sugalski
At 2:48 PM -0800 1/23/03, Brent Dax wrote: Dan Sugalski: # At 12:10 AM -0800 1/23/03, Brent Dax wrote: # Dan Sugalski: # # Since it looks like it's time to extend the packfile # format and the # # in-memory bytecode layout, this would be the time to start # discussing # # metadata. What sorts of

RE: Bytecode metadata

2003-01-23 Thread James Michael DuPont
--- Brent Dax [EMAIL PROTECTED] wrote: Dan Sugalski: # At 12:10 AM -0800 1/23/03, Brent Dax wrote: # Dan Sugalski: # # Since it looks like it's time to extend the packfile # format and the # # in-memory bytecode layout, this would be the time to start # discussing # # metadata. What

Re: Bytecode metadata

2003-01-23 Thread Leopold Toetsch
Dave Mitchell wrote: On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged in such a way that it (or most of it) can simply be mmap-ed in (RO),

Re: Bytecode metadata

2003-01-23 Thread Leopold Toetsch
Dan Sugalski wrote: Since it looks like it's time to extend the packfile format and the in-memory bytecode layout, this would be the time to start discussing metadata. What sorts of metadata do people think are useful to have in either the packfile (on disk) or in the bytecode (in memory).

Re: Bytecode metadata

2003-01-23 Thread Dan Sugalski
At 7:23 AM +0100 1/24/03, Leopold Toetsch wrote: Dave Mitchell wrote: On Thu, Jan 23, 2003 at 09:21:45PM +0100, Juergen Boemmels wrote: My current idea for the in memory format of the bytecode is this: I would strongly urge any file-based byte-code format to arranged in such a way that it

Re: Bytecode metadata

2003-01-22 Thread James Michael DuPont
--- Dan Sugalski [EMAIL PROTECTED] wrote: Since it looks like it's time to extend the packfile format and the in-memory bytecode layout, this would be the time to start discussing metadata. What sorts of metadata do people think are useful to have in either the packfile (on disk) or in