On Mon, May 4, 2009 at 5:51 PM, shire <sh...@php.net> wrote:
> Arnaud Le Blanc wrote:
>>
>> Hi,
>> On Mon, May 4, 2009 at 9:36 AM, shire<sh...@php.net> wrote:
>>>
>>> Regarding the ZEND_MMAP_AHEAD issue and the temp. fix that Dmitry put in
>>> we
>>> need to find a solution to that, perhaps I can play with that this week
>>> too
>>> as I think I'm seeing some related issues in my testing of 5.3.
>>> Essentially
>>> we abuse ZEND_MMAP_AHEAD by adding it to the file size and passing it to
>>> the
>>> mmap call which isn't at all valid and only really works up to PAGESIZE.
>>> We
>>> could possibly use YYFILL to re-allocate more space as necessary past the
>>> end of file to fix this.
>>
>> I was thinking of doing something like that with YYFILL too. However
>> there is a bunch of pointers to take in to account and to update (e.g.
>> yy_marker, yy_text, etc).
>>
>
> Yeah, I'm pretty sure that's how most of the example re2c code is setup:
>
> #define YYFILL(n) {cursor = fill(s, cursor);}
>
> uchar *fill(Scanner *s, uchar *cursor){
> if(!s->eof){
> unint cnt = s->lim - s->tok;
> uchar *buf = malloc((cnt + 1)*sizeof(uchar));
> memcpy(buf, s->tok, cnt);
> cursor = &buf[cursor - s->tok];
> s->pos = &buf[s->pos - s->tok];
> s->ptr = &buf[s->ptr - s->tok];
> s->lim = &buf[cnt];
> s->eof = s->lim; *(s->eof)++ = '\n';
> s->tok = buf;
> }
> return cursor;
> }
>
>
>
> -shire
>
This is what I seen too, but this is not always applicable. The
scanner have code that refers to yy_text, yy_start, yy_cursor,
yy_marker, etc. All those pointers point to the original buffer and
must be updated by fill(). At each point in time the scanner may
rollback to yy_marker or a rule may want to fetch yy_text or yy_start
at any time. So the buffer must be large enough to contain all data
from min(all_of_them) to max(all_of_them). That makes things a little
complicated and potentially less efficient than a big buffer for the
whole file.
Regards,
Arnaud
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
