Re: IMCC Reentarancy
Vishal, 在 2006/7/16 下午 11:57 時，Vishal Soni 寫到： a. Remove flex and implement re2c b. Remove static and global variables Now that the flex part is done, are you still willing to help removing the remaining static/global state? Apart from this we also need to refactor the code to get rid of arrays to a hash table implementation for macros. This would rock, too. Thanks, Audrey PGP.sig Description: This is a digitally signed message part
Re: IMCC Reentarancy
4th Option: fix flex. ;) -J -- On Sun, Jul 16, 2006 at 10:57:07PM -0500, Vishal Soni wrote: Hi, Please disregard the previous mail. Hit the wrong shortcut key!! I have been working on trying to make reenterant and/or thread-safe. There are couple of things that have come up which might make it difficult to make the existing implementation thread-safe/re-entrant. The current implementation is implemented using Flex and YACC. Flex implementation has limitations in C mode. The C lexer generated by flex cannot be reentrant/threadsafe. Flex generates thread-safe parsers only in C++ mode. This limition of flex will defeat the whole effort of removing global variables from IMCC. In my opinion if we cannot get global variable free code from flex there is no sense in proceeding with cleaning up the other global variables. Audrey Tang recommended using re2c as an alternative to flex (Lemon Parser replacement for Yacc). re2c generates reenterant/thread-safe parsers. I also spend some time reading up on the paper published with re2c. Initial indicators are that it produces scanners that run faster than flex. So here are some options that I have come up with that we have. I would like you guys and especially Allison and Chip to provide some feedback on how to proceed further. 1st Option: Hack it and patch it to death !!! --- Since flex is not generating reeentrant code, this option will get rid of flex altogether and replace it with re2c. This would require significant reworking on the code. So the plan of action would be as follows: a. Remove flex and implement re2c b. Remove static and global variables Apart from this we also need to refactor the code to get rid of arrays to a hash table implementation for macros. All in all this would be over hauling lot of code. 2nd: Inaction is the best action !!! --- Lets not do anything a leave the code as it is. Just say IMCC is not re-entrant/thread-safe and leave it there We will address this issue in future. I highly doubt it this is the route we want to take 3rd Option: Back to drawing board !!! This option would require a complete re-write of IMCC ( possibly could call it PIRC). The cons of this approach is we will have to re-implement the whole IMCC again. The programming languages will have to live with IMCC limitations as long as the new version is ready. The pros of this approach are a. A clean implementation rather than a prototypish implementation b. Make PIR compiler production release ready. The way the compiler sits right now it is not a good release candidate. c. Structure the code in a way that is easy to maintain and extend. The 3rd option is lot of work but might be a good option in the long run. These are just some of my thoughts. Please let me know what you guys think or do you have other options in mind. Whatever it is we need to come to consensus to make the IMCC reenterany a reality. As usual please provide feedback. -- Thanks, Vishal On 7/16/06, Vishal Soni [EMAIL PROTECTED] wrote: Hi, I have been working on trying to make reenterant and/or thread-safe. There are couple of things that have come up which might make it difficult to make the existing implemention thread-safe/re-entrant. The current implementation is implemented using Flex and YACC. Flex implementation has limitations in C mode. The C lexer generated by flex cannot be reentrant/threadsafe. Flex generates thread-safe parsers only in C++ mode. This limation of flex will defeat the whole effort of removing global variables from IMCC. In my opinion if we cannot get global variable free code from flex there is no sense in proceeding with cleaning up the other global variables. Audrey Tang reccomended using re2c as an alternative to flex (Lemon Parser replacemet for Yacc). re2c generates reenterant/thread-safe parsers. I also spend some time reading up on the paper published with re2c. Initial indicators are that it produces scanners that run faster than flex. So here are some options that I have come up with that we have. I would like you guys and especially Allison and Chip to provide some feedback on how to proceed further. 1st Option: Hack it and patch it to death !!! --- Since flex is not generating re-eentrant code, this option will get rid of flex altogether and replace it with re2c. This would require significant reworking on the code. So the plan of action would be as follows: a. Remove flex and implement re2c 2nd: Inaction is the best action !!! --- 3rd Option: Back to drawing board !!! -- Thanks, Vishal --
Re: IMCC Reentarancy
On 17 Jul 2006, at 05:22, Audrey Tang wrote: 在 2006/7/16 下午 11:57 時，Vishal Soni 寫到： I think that the lemon+re2c, being the more modern parsing tools, will make refactoring/hacking considerably easier. For future reference Ragel is definitely worth a look too: http://www.cs.queensu.ca/~thurston/ragel/ Cheers, Sam Phillips
Re: IMCC Reentarancy
在 2006/7/17 下午 3:41 時，Joshua Hoblitt 寫到： 4th Option: fix flex. ;) Turns out flex 2.5.30+ has a reentrant mode. However, it also has an incompatible API with earlier flex, even in non-reentrant mode. I've attached a patch under http:// rt.perl.org/rt3//Public/Bug/Display.html?id=34669 (need flex 2.5.30+ to run) that updates imcc.l to deal with %option reentrancy and the additional yyscanner parameter. However, imcc.y currently only allows one additional param to be passed as YYLEX_PARAM, and it's already taken by the Parrot interpreter, so until we put yyscanner into the interp somehow, or change the generated bison parser, this wouldn't work. As I'm writing this, I noticed that Allison has ruled that we go with PIR/PGE and eventually C-based libpge instead -- since a lexer refactoring that doesn't affect the IMCC API will somehow throw important projects on Parrot into a dead stall, and thread safety for PIR compilation is not a 1.0 goal anyway -- I'll abandon working on this, and focus on helping getting a C-based libpge started instead. :-) Thanks, Audrey PGP.sig Description: This is a digitally signed message part
IMCC Reentarancy
Hi, I have been working on trying to make reenterant and/or thread-safe. There are couple of things that have come up which might make it difficult to make the existing implemention thread-safe/re-entrant. The current implementation is implemented using Flex and YACC. Flex implementation has limitations in C mode. The C lexer generated by flex cannot be reentrant/threadsafe. Flex generates thread-safe parsers only in C++ mode. This limation of flex will defeat the whole effort of removing global variables from IMCC. In my opinion if we cannot get global variable free code from flex there is no sense in proceeding with cleaning up the other global variables. Audrey Tang reccomended using re2c as an alternative to flex (Lemon Parser replacemet for Yacc). re2c generates reenterant/thread-safe parsers. I also spend some time reading up on the paper published with re2c. Initial indicators are that it produces scanners that run faster than flex. So here are some options that I have come up with that we have. I would like you guys and especially Allison and Chip to provide some feedback on how to proceed further. 1st Option: Hack it and patch it to death !!! --- Since flex is not generating re-eentrant code, this option will get rid of flex altogether and replace it with re2c. This would require significant reworking on the code. So the plan of action would be as follows: a. Remove flex and implement re2c 2nd: Inaction is the best action !!! --- 3rd Option: Back to drawing board !!! -- Thanks, Vishal
Re: IMCC Reentarancy
Hi, Please disregard the previous mail. Hit the wrong shortcut key!! I have been working on trying to make reenterant and/or thread-safe. There are couple of things that have come up which might make it difficult to make the existing implementation thread-safe/re-entrant. The current implementation is implemented using Flex and YACC. Flex implementation has limitations in C mode. The C lexer generated by flex cannot be reentrant/threadsafe. Flex generates thread-safe parsers only in C++ mode. This limition of flex will defeat the whole effort of removing global variables from IMCC. In my opinion if we cannot get global variable free code from flex there is no sense in proceeding with cleaning up the other global variables. Audrey Tang recommended using re2c as an alternative to flex (Lemon Parser replacement for Yacc). re2c generates reenterant/thread-safe parsers. I also spend some time reading up on the paper published with re2c. Initial indicators are that it produces scanners that run faster than flex. So here are some options that I have come up with that we have. I would like you guys and especially Allison and Chip to provide some feedback on how to proceed further. 1st Option: Hack it and patch it to death !!! --- Since flex is not generating reeentrant code, this option will get rid of flex altogether and replace it with re2c. This would require significant reworking on the code. So the plan of action would be as follows: a. Remove flex and implement re2c b. Remove static and global variables Apart from this we also need to refactor the code to get rid of arrays to a hash table implementation for macros. All in all this would be over hauling lot of code. 2nd: Inaction is the best action !!! --- Lets not do anything a leave the code as it is. Just say IMCC is not re-entrant/thread-safe and leave it there We will address this issue in future. I highly doubt it this is the route we want to take 3rd Option: Back to drawing board !!! This option would require a complete re-write of IMCC ( possibly could call it PIRC). The cons of this approach is we will have to re-implement the whole IMCC again. The programming languages will have to live with IMCC limitations as long as the new version is ready. The pros of this approach are a. A clean implementation rather than a prototypish implementation b. Make PIR compiler production release ready. The way the compiler sits right now it is not a good release candidate. c. Structure the code in a way that is easy to maintain and extend. The 3rd option is lot of work but might be a good option in the long run. These are just some of my thoughts. Please let me know what you guys think or do you have other options in mind. Whatever it is we need to come to consensus to make the IMCC reenterany a reality. As usual please provide feedback. -- Thanks, Vishal On 7/16/06, Vishal Soni [EMAIL PROTECTED] wrote: Hi, I have been working on trying to make reenterant and/or thread-safe. There are couple of things that have come up which might make it difficult to make the existing implemention thread-safe/re-entrant. The current implementation is implemented using Flex and YACC. Flex implementation has limitations in C mode. The C lexer generated by flex cannot be reentrant/threadsafe. Flex generates thread-safe parsers only in C++ mode. This limation of flex will defeat the whole effort of removing global variables from IMCC. In my opinion if we cannot get global variable free code from flex there is no sense in proceeding with cleaning up the other global variables. Audrey Tang reccomended using re2c as an alternative to flex (Lemon Parser replacemet for Yacc). re2c generates reenterant/thread-safe parsers. I also spend some time reading up on the paper published with re2c. Initial indicators are that it produces scanners that run faster than flex. So here are some options that I have come up with that we have. I would like you guys and especially Allison and Chip to provide some feedback on how to proceed further. 1st Option: Hack it and patch it to death !!! --- Since flex is not generating re-eentrant code, this option will get rid of flex altogether and replace it with re2c. This would require significant reworking on the code. So the plan of action would be as follows: a. Remove flex and implement re2c 2nd: Inaction is the best action !!! --- 3rd Option: Back to drawing board !!! -- Thanks, Vishal -- Thanks, Vishal
Re: IMCC Reentarancy
在 2006/7/16 下午 11:57 時，Vishal Soni 寫到： a. A clean implementation rather than a prototypish implementation I think that the lemon+re2c, being the more modern parsing tools, will make refactoring/hacking considerably easier. Whilst you are converting the current IMCC implementation into the new toolchain, you'll be in the best position to find out inconsistencies, to-be- deprecated spots, as well as best strategies to hack in new features, such as nested expressions and composable macros. So I'd regard the lemon+re2c refactoring as a good preparation step (synchronize with the PIR mindset) before a full-fledged rewrite. Once that's in place, the rewrite may be reachable via a set of gradual refactor+deprecation cycle, which will make transition much, much easier. Thanks, Audrey PGP.sig Description: This is a digitally signed message part
