Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
Leopold Toetsch [EMAIL PROTECTED] writes: JüRgen BöMmels [EMAIL PROTECTED] wrote: There are still some things to do: - a global hash is still not totally right. It should be moved to the interpreter structure As Dan already outlined, macros should have some limited scope, probably per file. So currently the hash has to be cleared like the other (still) global strutures. Yes, it should be per-file scope. But the generated flex lexer has some more global state variables like yyin and yytext. Reworking the whole imcc global stuff isn't finished yet. Various structures have to be separated and kept per sub and or file. All of these should then be in IMCC_INFO hanging off from the interpreter structure. The IMCC_INFO does not need to be part of the interpreter. Anyway, I don't like to have a HASH and parrot Strings inside the lexer (when not really needed) just to manage macros. Currently a hash like that for SymRegs is just fine. It will be replaced by a still simple but expandable hash finally. Ok, granted. Parrot-Strings might be a little heavy. But introducing yet another hash implementation does not seem right to me either. It would be nice if both the PerlHash.pmc and the macro hash could use the same base implementation and use callbacks for the details. But on the other hand malloc/free and gc-memory have so diffrent requirements. bye boe
Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
Juergen Boemmels [EMAIL PROTECTED] wrote: Leopold Toetsch [EMAIL PROTECTED] writes: The IMCC_INFO does not need to be part of the interpreter. I don't like to have global state variables around. I don't know yet, what happens if 2 interpreter threads both do eval()... It will be replaced by a still simple but expandable hash finally. Ok, granted. Parrot-Strings might be a little heavy. But introducing yet another hash implementation does not seem right to me either. We need for sure a hash with plain cstrings as keys too. The code in hash.c seems rather heavy (and slow) to me, callbacks would slow down it additionally. bye boe leo
Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
On Thu, 16 Oct 2003, Leopold Toetsch wrote: Juergen Boemmels [EMAIL PROTECTED] wrote: Leopold Toetsch [EMAIL PROTECTED] writes: The IMCC_INFO does not need to be part of the interpreter. I don't like to have global state variables around. I don't know yet, what happens if 2 interpreter threads both do eval()... If it's per-interpreter, nothing, since each thread should have its own intepreter. (Though I can see some potential deadlock issues, though that's not specific to this) It will be replaced by a still simple but expandable hash finally. Ok, granted. Parrot-Strings might be a little heavy. But introducing yet another hash implementation does not seem right to me either. We need for sure a hash with plain cstrings as keys too. The code in hash.c seems rather heavy (and slow) to me, callbacks would slow down it additionally. It should all be black-box--if the code in hash.c is nasty, it can certainly be rewritten. (Though I'd keep in mind the need for a random seed in the hash, and Unicode makes hashing somewhat interesting) Dan
Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
At 10:21 AM 10/16/2003 -0400, Dan Sugalski wrote: On Thu, 16 Oct 2003, Leopold Toetsch wrote: Juergen Boemmels [EMAIL PROTECTED] wrote: Leopold Toetsch [EMAIL PROTECTED] writes: The IMCC_INFO does not need to be part of the interpreter. I don't like to have global state variables around. I don't know yet, what happens if 2 interpreter threads both do eval()... If it's per-interpreter, nothing, since each thread should have its own intepreter. (Though I can see some potential deadlock issues, though that's not specific to this) I don't like the idea of IMCC becoming so tightly coupled with the interpreter. I still wish to be able to build it standalone. Hopefully we can keep this in mind going forward. -Melvin
Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
Melvin Smith [EMAIL PROTECTED] wrote: I don't like the idea of IMCC becoming so tightly coupled with the interpreter. I still wish to be able to build it standalone. Hopefully we can keep this in mind going forward. IMCC_INFO is a macro, easily changable. But building it standalone would need some additional defines for PBC building and and the compiler callbacks. Anyway, why would you want to build it w/o parrot? The imcc files are supposed to move down one directory soon. There is no imcc$(EXE) anymore. Your baby evolved - its parrot :) -Melvin leo
Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
On Oct 16, 2003, at 7:21 AM, Dan Sugalski wrote: On Thu, 16 Oct 2003, Leopold Toetsch wrote: Juergen Boemmels [EMAIL PROTECTED] wrote: Leopold Toetsch [EMAIL PROTECTED] writes: The IMCC_INFO does not need to be part of the interpreter. I don't like to have global state variables around. I don't know yet, what happens if 2 interpreter threads both do eval()... If it's per-interpreter, nothing, since each thread should have its own intepreter. (Though I can see some potential deadlock issues, though that's not specific to this) The lex-generated stuff that Juergen mentioned could be more of a problem, since it seems to store its state in globals (yytext, etc.), and that might be pretty tricky to fix. I've always worried about the apparent lack of thread-safety of such generated lexers/parsers when I've run across them before, and wondered why they don't pass around a context structure instead. (I suppose the tools are old enough that they may actually pre-date threading as a widespread concept) JEff
[perl #24224] [PATCH] IMCC: Macros are handled via hash
# New Ticket Created by Jürgen Bömmels
# Please include the string: [perl #24224]
# in the subject line of all future correspondence about this issue.
# URL: http://rt.perl.org/rt2/Ticket/Display.html?id=24224
Hi,
Back in Feburary, when I hacked up macro support for IMCC I
implemented the macro_table as a static preallocated array of fixed
size and used a linear search for lookup. Now I have took the time and
transformed it to use a HASH.
The macros are now stored in a global HASH named macro_hash. They are
indexed by their name. So the lookup of a macro is a simple hash_get.
Furthermore i moved the inplace definition of the .constant rule to
a support function, the definition got quite fat.
There are still some things to do:
- a global hash is still not totally right. It should be moved to the
interpreter structure
- The size of the expansion is still limited in some way by a fixed
buffer of size 4096. This needs to go away.
Comments
bö
-- attachment 1 --
url: http://rt.perl.org/rt2/attach/66091/49389/4aa72c/macro_hash.diff
Index: languages/imcc/imcc.l
===
RCS file: /cvs/public/parrot/languages/imcc/imcc.l,v
retrieving revision 1.67
diff -u -r1.67 imcc.l
--- languages/imcc/imcc.l 14 Oct 2003 07:40:02 - 1.67
+++ languages/imcc/imcc.l 15 Oct 2003 16:59:31 -
@@ -15,6 +15,7 @@
#include string.h
#include imc.h
#include parser.h
+#include parrot/hash.h
#define MAX_PARAM 16
@@ -30,9 +31,9 @@
int line;
};
-/* XXX: boe: rework this hack to use a hash */
-struct macro_t macros[256];
-int num_macros = 0;
+/* XXX boe: This should not be global */
+HASH *macro_hash = 0;
+char *current_macro = 0;
char temp_buffer[4096];
@@ -58,6 +59,7 @@
static int yylex_skip (YYSTYPE *valp, void *interp, const char *skip);
static int read_macro (YYSTYPE *valp, void *interp);
+static int read_constant (YYSTYPE *valp, void *interp);
static int expand_macro (YYSTYPE *valp, void *interp, const char *name);
static void include_file (char *file_name);
static int in_pod;
@@ -242,31 +244,7 @@
}
emit.constant {
-int c;
- char *name;
- struct macro_t *m;
-
- BEGIN(macro);
- c = yylex_skip(valp, interp, );
- if (c != IDENTIFIER)
- fataly(EX_SOFTWARE, sourcefile, line,
- Constant names must be identifiers);
-
- name = str_dup(valp-s);
-
- c = yylex_skip(valp, interp, );
-if (c != INTC c != FLOATC c != STRINGC c != REG)
-fataly(EX_SOFTWARE, sourcefile, line,
-Constant '%s' value must be a number,
-stringliteral or register, name);
-
- m = macros + num_macros++;
- m-name = name;
- m-expansion = str_dup (valp-s);
- m-params.num_param = 0;
-
- BEGIN (emit);
- return MACRO;
+return read_constant(valp, interp);
}
*.include {
@@ -389,7 +367,6 @@
macro${ID}: return LABEL;
macro.local{WS}+ {
char *label;
- char *name = macros[num_macros].name;
if (yylex(valp, interp) != LABEL)
fataly(EX_SOFTWARE, sourcefile, line, LABEL expected);
@@ -397,8 +374,8 @@
if (valp) {
YYCHOP();
- label = mem_sys_allocate(strlen(name) + yyleng + 15);
- sprintf(label, local__%s__%s__$:, name, yytext+1);
+ label = mem_sys_allocate(strlen(current_macro) + yyleng + 15);
+ sprintf(label, local__%s__%s__$:, current_macro, yytext+1);
valp-s = label;
}
@@ -408,11 +385,10 @@
macro.${ID} {
char *label;
- char *name = macros[num_macros].name;
if (valp) {
- label = mem_sys_allocate(strlen(name) + yyleng + 15);
- sprintf(label, local__%s__%s__$, name, yytext+2);
+ label = mem_sys_allocate(strlen(current_macro) + yyleng + 15);
+ sprintf(label, local__%s__%s__$, current_macro, yytext+2);
valp-s = label;
}
@@ -570,8 +546,13 @@
read_macro (YYSTYPE *valp, void *interp)
{
int c;
-struct macro_t *m = macros + num_macros;
+struct macro_t *m = mem_sys_allocate(sizeof(struct macro_t));
+HASH_ENTRY value;
+STRING *key;
int start_cond;
+char *last_macro = current_macro;
+
+if (!macro_hash) new_hash(interp, macro_hash);
temp_buffer[0]='\0';
@@ -586,6 +567,8 @@
m-name = valp-s;
m-line = line;
+current_macro = valp-s;
+
/* white space is allowed between macro and opening paren) */
c = yylex_skip(valp, interp, );
@@ -609,11 +592,56 @@
}
free(valp-s);
+current_macro = last_macro;
+
BEGIN(start_cond);
-macros[num_macros].expansion = str_dup(temp_buffer);
+m-expansion = str_dup(temp_buffer);
+
+key = string_from_cstring(interp, m-name, 0);
+value.type = enum_type_PMC;
+value.val.pmc_val = (PMC*)m;
+hash_put(interp, macro_hash, key, value);
+
+return MACRO;
+}
+
+static int
+read_constant (YYSTYPE *valp, void *interp)
+{
+int c;
+char *name;
+struct macro_t *m;
+STRING *key;
+HASH_ENTRY value;
+
+if (!macro_hash)
Re: [perl #24224] [PATCH] IMCC: Macros are handled via hash
JüRgen BöMmels [EMAIL PROTECTED] wrote: There are still some things to do: - a global hash is still not totally right. It should be moved to the interpreter structure As Dan already outlined, macros should have some limited scope, probably per file. So currently the hash has to be cleared like the other (still) global strutures. Reworking the whole imcc global stuff isn't finished yet. Various structures have to be separated and kept per sub and or file. All of these should then be in IMCC_INFO hanging off from the interpreter structure. Anyway, I don't like to have a HASH and parrot Strings inside the lexer (when not really needed) just to manage macros. Currently a hash like that for SymRegs is just fine. It will be replaced by a still simple but expandable hash finally. bö leo
