Re: [PHP-DEV] [RFC] Generators
Morgan L. Owens wrote: I accept your point about not caring about how the data was created, but on the other side, if the data creation is handling a lot more data than the consumer needs there is an amount of processing time that is wasted. The quick way of doing something does not equate to the best way of doing it. Only if the producer does work unnecessary for determining the next datum required by the consumer. It doesn't have to create all the data at once (if it did you might as well stuff it all in a big array and use that). I think my only point here was that a 'generic' producer many benefit from additional work to make it more efficient, but it's only the consumer end that knows what it needs? This is the problem with a lot of the database 'abstraction' ... there are a lot of additional processes carried out to make the abstraction stuff work which could be optimised a lot easier if they weren't wrapped up away so well. The iterator/generator approach to work flow is going down the same path which is fine for some applications but not so flexible for others? BOTH approaches to workflow have a place, it's working out at what point one is better then the other? Currently we do not have any real basis to compare things :( -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Morgan L. Owens wrote: I suppose if you're lucky the consumer doesn't need to remember any state between one notification and the next, and it can treat every dataum like it's the first. But that is hardly the general case. Actually 75% of my 'producers' are simply reading the next line from a file. Which is why the example given in the RFC got my goat so much - I don't recall ever trying to read the whole file into an array ;) . Wrapping that in an iterator gives me nothing and neither does 'yield' I think. My 'producer' simply needs to open the file and read it ... either as a raw line of text or a CSV conversion. Which the 'consumer' part of the relevant object simply processes. The rest of my produces are data feeds from the database ... which already have a 'next' etc ... wrapping them in an iterator seems pointless as well? And essentially they already 'yield' ? ( Thanks for the links ... The php samples in sourcemaking.com look interesting ) -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: Re: [PHP-DEV] [RFC] Generators
On 2012-08-23 02:03, Lester Caine wrote: I accept your point about not caring about how the data was created, but on the other side, if the data creation is handling a lot more data than the consumer needs there is an amount of processing time that is wasted. The quick way of doing something does not equate to the best way of doing it. Only if the producer does work unnecessary for determining the next datum required by the consumer. It doesn't have to create all the data at once (if it did you might as well stuff it all in a big array and use that). -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: Re: [PHP-DEV] [RFC] Generators
On 2012-08-23 00:10, Lester Caine wrote: Then the next example is an 'iterator' ... which you are right ... I do not appreciate either, because they require an insane amount of overhead for what would be easy if the first example had been done right! I did try them, in the past, but the overhead outweighed any advantage and I can't find them in any of the projects I work with apart as a blank frame in ADOdb ... which nobody seems to use. But as far as I've been able to make out, your method _still_ has the overhead of maintaining state during the iteration - only now it's each consumer's responsibility to remember its state between successive notifications by the producer. The producer can't help with that, because each consumer has different maintenance requirements; managing the iteration process has therefore been split across two distinct objects with the consumer doing the hard part (which is why the producer looks so simple). I suppose if you're lucky the consumer doesn't need to remember any state between one notification and the next, and it can treat every dataum like it's the first. But that is hardly the general case. To describe iterators further, here are a few links (note that they're not necessarily the best - they're just the first non-Wikipedia, non-Amazon links I got when Googling for Design Patterns): http://www.dofactory.com/Patterns/PatternIterator.aspx http://www.oodesign.com/iterator-pattern.html http://sourcemaking.com/design_patterns/iterator It might also help to compare the Observer pattern, in the special case where the one-many relationship is actually one-one. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Morgan L. Owens wrote:
As I wrote in an earlier post:
On 2012-08-09 15:30, Morgan L. Owens wrote:
I for one am lazy, and would much prefer writing:
?php
function append_iterator($first, $second)
{
foreach($first as $i)
{
yield $i;
}
foreach($second as $i)
{
yield $i;
}
}
?
Morgan
I think THIS is an nice example of why I am having a problem with the concept.
EXACTLY what is yield doing here?
If this was an SQL procedure, then at the first yield (suspend in SQL), the
procedure returns the values and the external process uses them. Next call to
the procedure returns the next set of values.
In your example you have TWO 'suspend' points, and this is what is making things
confusing for me. SO is this returning every $first value followed by every
$second ?
Without some better explanation of how the work flows, reading and understanding
even what the above example means is a problem?
The answer may be simple but people telling me 'just because you don't
understand is no reason to reject them' is just irritating. I'd LIKE to
understand what they are intended to provide but I'm still just not getting it
as hopefully the explanation here highlights?
I'm sure I'm not alone here?
--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Wed, Aug 22, 2012 at 10:21 AM, Lester Caine les...@lsces.co.uk wrote:
Morgan L. Owens wrote:
As I wrote in an earlier post:
On 2012-08-09 15:30, Morgan L. Owens wrote:
I for one am lazy, and would much prefer writing:
?php
function append_iterator($first, $second)
{
foreach($first as $i)
{
yield $i;
}
foreach($second as $i)
{
yield $i;
}
}
?
Morgan
I think THIS is an nice example of why I am having a problem with the
concept.
EXACTLY what is yield doing here?
If this was an SQL procedure, then at the first yield (suspend in SQL),
the procedure returns the values and the external process uses them. Next
call to the procedure returns the next set of values.
In your example you have TWO 'suspend' points, and this is what is making
things confusing for me. SO is this returning every $first value followed
by every $second ?
Without some better explanation of how the work flows, reading and
understanding even what the above example means is a problem?
The answer may be simple but people telling me 'just because you don't
understand is no reason to reject them' is just irritating. I'd LIKE to
understand what they are intended to provide but I'm still just not getting
it as hopefully the explanation here highlights?
I'm sure I'm not alone here?
if you are a consumer of a generator, you don't have to understand the
underlying process, as it 'just works'.
you don't have to know about how a specific iterator or generator works
internally as long as they are implementing the iterator and countable
interface you can just use them as arrays.
of course if you want to write your own generators, you have to understand
how those work, but I think that after this much discussion and
explanations, we can't really help you with that.
why is it matter that there is two place in the function body with yield?
you can have methods with two break, or continue.
does that also confuse you?
it doesn't make a difference, if you yield only once, and put that into a
loop, or copy paste the yield line ten times.
yield always does the same thing, pass the execution to the caller.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
Re: [PHP-DEV] [RFC] Generators
Ferenc Kovacs wrote: it doesn't make a difference, if you yield only once, and put that into a loop, or copy paste the yield line ten times. yield always does the same thing, pass the execution to the caller. I KNOW that I am sounding thick here but you don't have to understand the underlying process, as it 'just works'. is exactly where I'm having the problem. Perhaps because some of these things ARE treated as 'magic' ? But I need to understand the work flow if I'm going to profile it and optimise use of this stuff. When working on data feeds from the database, HOW is everything in making it work at it's fastest. One way of doing things can be several orders of magnitude slower than another. In this example, if I look at it as an SQL stored procedure, on every 'pass back to the caller' I get the next piece of data, so I'm getting a sequence of all the 'first' objects, followed by all the 'second' objects. What is consuming that data just takes what is given each time and processes it. Somewhere in this magic, the data is packaged so it can be passed, and it's yield that is packaging the return. In SQL that return would be a database record each time, with no relation to previous or next record, but I presume talk of 'auto-increment' is to automatically add a record count? And talk of 'returning arrays' only applies if 'yield' specifically outputs the next array element? It's this 'just looks like an array' that is getting me since the whole point is NOT to return an array of objects? In my method of working I can either build an array of records from the data feed, or simply handle each item in a sequence and throw it away. You don't want to build the array was the reason given for needing generators? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Wed, Aug 22, 2012 at 12:11 PM, Lester Caine les...@lsces.co.uk wrote: Ferenc Kovacs wrote: it doesn't make a difference, if you yield only once, and put that into a loop, or copy paste the yield line ten times. yield always does the same thing, pass the execution to the caller. I KNOW that I am sounding thick here but you don't have to understand the underlying process, as it 'just works'. is exactly where I'm having the problem. Perhaps because some of these things ARE treated as 'magic' ? But I need to understand the work flow if I'm going to profile it and optimise use of this stuff. When working on data feeds from the database, HOW is everything in making it work at it's fastest. One way of doing things can be several orders of magnitude slower than another. do you know how the goto, break, continue throw, etc. contstructs work? from a userland developer point of view, the whole generator stuff is explained in these few lines: Generators work by passing control back and forth between the generator and the calling code: When you first call the generator function ($lines = getLinesFromFile($fileName)) the passed argument is bound, but nothing of the code is actually executed. Instead the function directly returns a Generator object. That Generator object implements the Iteratorinterface and is what is eventually traversed by the foreach loop: Whenever the Iterator::next() method is called PHP resumes the execution of the generator function until it hits a yield expression. The value of that yield expression is what Iterator::current() then returns. when you first call a generator function, it will return an instance of the Generator class, which you can iterate over, which in turn will execute the method body yield-by-yield (until the generator is depleted or you stop iterating it). In this example, if I look at it as an SQL stored procedure, on every 'pass back to the caller' I get the next piece of data, so I'm getting a sequence of all the 'first' objects, followed by all the 'second' objects. What is consuming that data just takes what is given each time and processes it. Somewhere in this magic, the data is packaged so it can be passed, and it's yield that is packaging the return. In SQL that return would be a database record each time, with no relation to previous or next record, but I presume talk of 'auto-increment' is to automatically add a record count? And talk of 'returning arrays' only applies if 'yield' specifically outputs the next array element? It's this 'just looks like an array' that is getting me since the whole point is NOT to return an array of objects? In my method of working I can either build an array of records from the data feed, or simply handle each item in a sequence and throw it away. You don't want to build the array was the reason given for needing generators? I can't really follow your sql example, but let's try this another way: You understand how Iterators work, right? ( http://php.net/manual/en/language.oop5.iterations.php http://www.php.net/manual/en/class.iterator.php) The proposed generator implementation works the same way from the consumer/caller perspective. They call a function, which returns them a Generator object, which implements the iterator interface, so the caller can iterate over the returned object. In the background, the original function body acts as the implementation of the next() method of the iterator inteface, and the yield keyword is used to mark the end of each next() call. Does this make it more clear to you? -- Ferenc Kovács @Tyr43l - http://tyrael.hu
Re: [PHP-DEV] [RFC] Generators
Ferenc Kovacs wrote: I can't really follow your sql example, but let's try this another way: You understand how Iterators work, right? (http://php.net/manual/en/language.oop5.iterations.php http://www.php.net/manual/en/class.iterator.php) The proposed generator implementation works the same way from the consumer/caller perspective. They call a function, which returns them a Generator object, which implements the iterator interface, so the caller can iterate over the returned object. In the background, the original function body acts as the implementation of the next() method of the iterator inteface, and the yield keyword is used to mark the end of each next() call. Does this make it more clear to you? I've just gone back over the rfc and I probably understand now why I was getting confused. Actually it really does irritate that the first 'example' on the page is simply the wrong way of doing it anyway. We should not be propagating bad code like that and using it as justification for something new just got my back up from the start. The extra Why not just use callback functions? irritates more because again it's another poor example. Drop the callback altogether and just add a call to process the data using a suitable object ... Then the next example is an 'iterator' ... which you are right ... I do not appreciate either, because they require an insane amount of overhead for what would be easy if the first example had been done right! I did try them, in the past, but the overhead outweighed any advantage and I can't find them in any of the projects I work with apart as a blank frame in ADOdb ... which nobody seems to use. So I'm working from the wrong base simply because interators also introduced too much overhead for no gain. Now we are trying another fix to make iterators work better? I suppose the next step is simply using 'yield' without iterators? The iterator magic is only used to hide some of the process flow where a simple object might work better. I CAN actually see where the use of yield might work, but I am probably simply to stuck in my ways. Niktia's A concrete example, which was actually my initial motivation to write the generators patch: is probably another good example of why iterators are simply wrong more times than they are right? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
I've just gone back over the rfc and I probably understand now why I was getting confused. Actually it really does irritate that the first 'example' on the page is simply the wrong way of doing it anyway. We should not be propagating bad code like that and using it as justification for something new just got my back up from the start. The extra Why not just use callback functions? irritates more because again it's another poor example. Drop the callback altogether and just add a call to process the data using a suitable object ... Then the next example is an 'iterator' ... which you are right ... I do not appreciate either, because they require an insane amount of overhead for what would be easy if the first example had been done right! I did try them, in the past, but the overhead outweighed any advantage and I can't find them in any of the projects I work with apart as a blank frame in ADOdb ... which nobody seems to use. So I'm working from the wrong base simply because interators also introduced too much overhead for no gain. Now we are trying another fix to make iterators work better? I suppose the next step is simply using 'yield' without iterators? The iterator magic is only used to hide some of the process flow where a simple object might work better. I CAN actually see where the use of yield might work, but I am probably simply to stuck in my ways. Niktia's A concrete example, which was actually my initial motivation to write the generators patch: is probably another good example of why iterators are simply wrong more times than they are right? So now you understand how the proposed Generator implementation works? If you think that Iterators require a bunch of boilerplate code, I can understand that, and that was one of the motivations behind the Generators rfc I guess. If you think that using Iterators (hence reversing the process control between the consumer and producer) is always wrong, I cannot agree with that. There are cases when you don't care about how the data get created/filtered, you just want to iterate over it, without the need to build a huge array and only start working on it, when the whole thing is ready. Your style of processing data is tightly coupling the generation of the data and the processing of the data, which works for you, but in some scenarios and for some people it is a bad architectural design. -- Ferenc Kovács @Tyr43l - http://tyrael.hu
Re: [PHP-DEV] [RFC] Generators
On Wed, Aug 22, 2012 at 2:10 PM, Lester Caine les...@lsces.co.uk wrote: Ferenc Kovacs wrote: I can't really follow your sql example, but let's try this another way: You understand how Iterators work, right? (http://php.net/manual/en/language.oop5.iterations.php http://www.php.net/manual/en/class.iterator.php) The proposed generator implementation works the same way from the consumer/caller perspective. They call a function, which returns them a Generator object, which implements the iterator interface, so the caller can iterate over the returned object. In the background, the original function body acts as the implementation of the next() method of the iterator inteface, and the yield keyword is used to mark the end of each next() call. Does this make it more clear to you? I've just gone back over the rfc and I probably understand now why I was getting confused. Actually it really does irritate that the first 'example' on the page is simply the wrong way of doing it anyway. We should not be propagating bad code like that and using it as justification for something new just got my back up from the start. The extra Why not just use callback functions? irritates more because again it's another poor example. Drop the callback altogether and just add a call to process the data using a suitable object ... Then the next example is an 'iterator' ... which you are right ... I do not appreciate either, because they require an insane amount of overhead for what would be easy if the first example had been done right! I did try them, in the past, but the overhead outweighed any advantage and I can't find them in any of the projects I work with apart as a blank frame in ADOdb ... which nobody seems to use. So I'm working from the wrong base simply because interators also introduced too much overhead for no gain. Now we are trying another fix to make iterators work better? I suppose the next step is simply using 'yield' without iterators? The iterator magic is only used to hide some of the process flow where a simple object might work better. I CAN actually see where the use of yield might work, but I am probably simply to stuck in my ways. Niktia's A concrete example, which was actually my initial motivation to write the generators patch: is probably another good example of why iterators are simply wrong more times than they are right? Hi Lester! You may want to look through this presentation: http://www.dabeaz.com/generators/Generators.pdf The RFC is a technical document outlining the details of the implementation; it isn't well suited as an entry-level tutorial on the topic. The above presentation might do a better job explaining the concept and its uses. But I think I understood your main problem with generators now. Generators are very useful for the creation of reusable and pluggable components. You don't seem particularly fond of that kind of programming and prefer hardcoded and tightly coupled implementations. In that case generators probably won't give you anything. But they won't hurt you either ;) You can still practice the One Big Method style of programming. Thanks, Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Ferenc Kovacs wrote:
So now you understand how the proposed Generator implementation works?
If you think that Iterators require a bunch of boilerplate code, I can
understand that, and that was one of the motivations behind the Generators rfc I
guess.
If you think that using Iterators (hence reversing the process control between
the consumer and producer) is always wrong, I cannot agree with that.
There are cases when you don't care about how the data get created/filtered, you
just want to iterate over it, without the need to build a huge array and only
start working on it, when the whole thing is ready.
Having been working with databases for 20+ years one gets used to ensuring only
the required data is accessed in the first place ;) Invariably I'm working with
a cursor on a data source of some sort - so no big array.
Your style of processing data is tightly coupling the generation of the data and
the processing of the data, which works for you, but in some scenarios and for
some people it is a bad architectural design.
I think this is a good example of where 'you don't need to use it' is just the
wrong attitude. There are good and bad examples of everything because there are
many ways of doing things. PHP IS just too flexible. This is why I keep banging
on about 'examples of good coding practice' - heck *I* need them so that I can
tidy up 10 years worth of code! From my point of view the generators rfc is
simply not presenting the case in a good light and even reading it today, the
'comparisons' against what is already bad code grate :( I'd not be doing that in
the first case comes to mind ... so switch off.
Working out the flow through
$lines = getLinesFromFile($fileName);
foreach ($lines as $line) {
// do something with $line
}
Where yield is used in 'getLinesFromFile' still does not look logical to me, and
if we break out of the foreach loop just how does '$lines' get closed out later?
I am assuming that the 'getLinesFromFile' is being actioned once for each cycle
of the foreach loop so has to keep it's state, it's not being able to 'see' the
magic that is the hidden -next() which is the problem. I think that THIS is why
I'd prefer to see a 'generator function' header so that straight away I know
there is some magic under the hood to do with THIS particular function. Scanning
a function and then finding a yield later then needs you to back wind and
understand that is IS an interruptible function. Generators just seem to be
hiding the boilerplate code that an interator needs to display?
I accept your point about not caring about how the data was created, but on the
other side, if the data creation is handling a lot more data than the consumer
needs there is an amount of processing time that is wasted. The quick way of
doing something does not equate to the best way of doing it.
--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Nikita Popov wrote: But I think I understood your main problem with generators now. Generators are very useful for the creation of reusable and pluggable components. You don't seem particularly fond of that kind of programming and prefer hardcoded and tightly coupled implementations. In that case generators probably won't give you anything. But they won't hurt you either;) You can still practice the One Big Method style of programming. Working with millions of data records, one keeps the interface between database and php as tidy as possible. So even some areas of ADOdb have been 'adjusted'. The majority of my code is 'reusable', just plugging in a new object where I need to support a new data structure. In practice I SHOULD probably cut out ADOdb altogether for even better performance, but any of my stuff can run on any of the other databases ADOdb supports out of the box. bitweaver was forked from tikiwiki simply because that was using the 'One Big Method' of programming and we wanted to make things a LOT more modular and reusable! -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Morgan L. Owens wrote:
For the third one ... I'm still waiting for some clarification on how
yield is SUPPOSED to work anyway? If you are using a 'generator' to
return a sequence of data elements, then just what does happen between
each call to the generator ... DOES the generator get called for each
value until something flags there are no more? I still don't see the
advantage of restructuring how things are called to get around a problem
which can just as easily be solved by proper design in the first place.
Have you even read the RFC yet? Calling a generator function returns an instance
of a Generator object. That object has a method interface - described in the
RFC. That interface is accessible to the foreach() construct so the loop can
iterate over the successive values returned by the object.
If by proper design you mean your own personal preference for using callbacks,
that's also discussed in the RFC. You were told this last month:
You see that is my problem ... WHY is using an existing object instance so
wrong?
$handle = fopen(../data/clientdatabase.csv, r);
if ( $handle == FALSE) {
$row = -999;
} else {
while (($data = fgetcsv($handle, 800, ,)) !== FALSE) {
if ( $row ) $contact-ContactRecordLoad( $data );
$row++;
}
fclose($handle);
}
$contact-ContactRecordLoad( $data ); simply processes the data and then throws
it away, so why is this so bad? I am just trying to understand why my long time
method of working is so frond upon, and understand if generators will improve
the performance of the above 'very heavy' code. ContactRecordLoad identifies the
'line type' and then extracts the data based on the line type. My NLPG data
importer has a dozen processor functions selected by the base 'RecordLoad',
normally the only problem area is translating different date formats into
something generic, so the basic iteration loop isn't a problem. How will the use
of 'yield' make this better? Or more important - based on the rfc - why is this
so wrong?
--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Tue, Aug 21, 2012 at 12:31 AM, Lester Caine les...@lsces.co.uk wrote:
Morgan L. Owens wrote:
For the third one ... I'm still waiting for some clarification on how
yield is SUPPOSED to work anyway? If you are using a 'generator' to
return a sequence of data elements, then just what does happen between
each call to the generator ... DOES the generator get called for each
value until something flags there are no more? I still don't see the
advantage of restructuring how things are called to get around a problem
which can just as easily be solved by proper design in the first place.
Have you even read the RFC yet? Calling a generator function returns an
instance
of a Generator object. That object has a method interface - described in
the
RFC. That interface is accessible to the foreach() construct so the loop
can
iterate over the successive values returned by the object.
If by proper design you mean your own personal preference for using
callbacks,
that's also discussed in the RFC. You were told this last month:
You see that is my problem ... WHY is using an existing object instance so
wrong?
$handle = fopen(../data/clientdatabase.**csv, r);
if ( $handle == FALSE) {
$row = -999;
} else {
while (($data = fgetcsv($handle, 800, ,)) !== FALSE) {
if ( $row ) $contact-ContactRecordLoad( $data );
$row++;
}
fclose($handle);
}
$contact-ContactRecordLoad( $data ); simply processes the data and then
throws it away, so why is this so bad? I am just trying to understand why
my long time method of working is so frond upon, and understand if
generators will improve the performance of the above 'very heavy' code.
ContactRecordLoad identifies the 'line type' and then extracts the data
based on the line type. My NLPG data importer has a dozen processor
functions selected by the base 'RecordLoad', normally the only problem area
is translating different date formats into something generic, so the basic
iteration loop isn't a problem. How will the use of 'yield' make this
better? Or more important - based on the rfc - why is this so wrong?
No one is frowning on this way of doing things. It's perfectly fine for
the circumstances. Imagine however, that you have a new client, who
provides a different format in their clientdatabase.csv. There are
definitely ways you can handle this in your current code. But with
generators you might be able to do:
function origFormat()
{
$handle = fopen(../data/clientdatabase.csv, r);
if ( $handle == FALSE) {
return false;
} else {
$row = fgetcsv($handle, 800, ,);
if($row !== false)
yield fgetcsv($handle, 800, ,);
}
}
function strangeFormat()
{
$data = origFormat();
foreach($data as $row)
{
//move some fields around to match the original format
yield $row;
}
}
if($client == 'strange')
$data = strangeFormat();
else
$data = origFormat();
foreach($data as $row)
{
$contact-ContactRecordLoad($data);
}
Another possibility is that you are importing records from another
database, and now you can create a third generator that pulls from the db.
The point is that your data source(the producer) is abstracted from the
consumer, so the consumer doesn't need to have the logic to handle the
different cases inside its (possibly complicated) foreach() loop. And the
producer can be a simple function, rather than an object that implements
the Iterator interface.
Again, the case you've cited is probably not a case where generators give
much advantage. But anytime the logic to produce some data becomes more
complicated (Imagine that you needed to skip some of the lines in the .csv
file based on some kind of filter), then being able to extricate the
producer logic from the consumer logic is helpful.
Thanks,
John
Re: [PHP-DEV] [RFC] Generators
John LeSueur wrote: Again, the case you've cited is probably not a case where generators give much advantage. But anytime the logic to produce some data becomes more complicated (Imagine that you needed to skip some of the lines in the .csv file based on some kind of filter), then being able to extricate the producer logic from the consumer logic is helpful. But in my 'method of working' one simply changes the '$contact-' object and loads the one that matches the data you are handling. Sage, paypal, moneybox, nlpg and so on. I've a dozen or so data models currently. skip some of the lines simply equates to saving or not some of the detail lines such as shopping basket details on paypal or property details in nlpg. The base RecordLoad takes care of that but in reality one always saves all of the data, and later access to the database provides any filtering required when processing results for the screen. I suppose that the 'producer logic' is populating the database, while the 'consumer logic' is later reading that data back - filtered as required. If I was keeping everything in memory things would probably be different ... but I still can't see why the work flow has to be reorganised rather than simply following the natural flow that the job requires. Adding 'yield' how ever it's added loads additional development work on all of the ancillary tools many of which still haven't caught up with the 5.3 extras let alone 5.4, and doesn't seem to provide any outstanding advantages? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Dearest Lester, I know you don't understand generators. You've posted multiple times about it. And you know what? It's okay; you don't have to understand them. I do understand generators and would LOVE to have them available for some of the things I do in my line of work; that's perfectly fine too. Your Friend, Levi Morrison -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: Re: [PHP-DEV] [RFC] Generators
On 2012-08-22 04:35, Lester Caine wrote: John LeSueur wrote: Again, the case you've cited is probably not a case where generators give much advantage. But anytime the logic to produce some data becomes more complicated (Imagine that you needed to skip some of the lines in the .csv file based on some kind of filter), then being able to extricate the producer logic from the consumer logic is helpful. But in my 'method of working' one simply changes the '$contact-' object and loads the one that matches the data you are handling. Sage, paypal, moneybox, nlpg and so on. I've a dozen or so data models currently. That is one of the differences. With the Iterator pattern that is being automated here, all the boilerplate of maintaining state between iteration steps is contained within the iterator object, and is the same for all uses of that object, so only needs to be written once (and, with this extension, that writing is automated). Using the Observer pattern like you do requires boilerplate to be written afresh for every distinct _use_ of the iterator. That boilerplate isn't directly related to the consumer's actual job but to the iteration process (limiting the range of producers that consumer can be fed from). but I still can't see why the work flow has to be reorganised rather than simply following the natural flow that the job requires. The adjective natural would only apply if you always see iteration as the producer pushing its productions out to the consumer (here - go work on this), and never picture the situation as the consumer pulling them from the producer (give me something to work on). Otherwise it's just a different perspective. The Observer pattern certainly does have its uses (otherwise it wouldn't be a pattern): if the things being produced are interpreted as events being handled (hey, a new line of CSV has just come in!), there may be multiple consumers interested in being notified when those events occur. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: Re: [PHP-DEV] [RFC] Generators
Oh, yes...
On 2012-08-22 04:35, Lester Caine wrote:
and doesn't seem to
provide any outstanding advantages?
As I wrote in an earlier post:
On 2012-08-09 15:30, Morgan L. Owens wrote:
I for one am lazy, and would much prefer writing:
?php
function append_iterator($first, $second)
{
foreach($first as $i)
{
yield $i;
}
foreach($second as $i)
{
yield $i;
}
}
?
to
?php
87 lines elided
?
I'm not making any assumption about where the generator returned by
append_iterator() is going to be used. In particular, it might not be in
a public object method or other callable. But its use would probably
look something like:
?php
foreach(append_iterator($left, $right) as $i)
{
... do something with $i.
}
?
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
hi Rasmus! On Mon, Aug 20, 2012 at 2:01 AM, Rasmus Lerdorf ras...@lerdorf.com wrote: I would still like to understand what this generator keyword would actually do. I don't see how it would work. Would a function marked generator somehow not be allowed to return normally or to finish and not return anything? How could this be enforced? I am completely against any keyword that is essentially documentation-only. It is not documentation only (or should not but I have to check the implementation). It allows the use of the generators features inside this function. Just like what we have for static methods and this (somehow). Also I would really not begin to totally rethink the way generators work or are defined. There are clean and known ways in other languages, the closer we get from them the easier it will be to learn how to use them in PHP. Cheers, -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 20/08/12 02:01, Rasmus Lerdorf wrote: I would still like to understand what this generator keyword would actually do. I don't see how it would work. Would a function marked generator somehow not be allowed to return normally or to finish and not return anything? How could this be enforced? I am completely against any keyword that is essentially documentation-only. -Rasmus Given that such function could return several times, seems a different enough function type to have its keyword. You could not decorate it and rely instead on the presence of the yield keyword, but parsers will thank knowing about it from the start rather than realising at mid-parsing that the function is a completely different beast. Regards -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 08/20/2012 07:56 AM, Ángel González wrote:
On 20/08/12 02:01, Rasmus Lerdorf wrote:
I would still like to understand what this generator keyword would
actually do. I don't see how it would work. Would a function marked
generator somehow not be allowed to return normally or to finish and not
return anything? How could this be enforced? I am completely against any
keyword that is essentially documentation-only.
-Rasmus
Given that such function could return several times, seems a different
enough function type to have its keyword.
You could not decorate it and rely instead on the presence of the yield
keyword, but parsers will thank knowing about it from the start rather
than realising at mid-parsing that the function is a completely
different beast.
So how about something like this:
generator function f() {
echo Hello World;
}
generator function f() {
return 1;
}
generator function f($arg) {
if(f!$arg) yield 1;
else if($arg0) return 1;
else return;
}
What does the generator keyword mean in each of these cases? Anything?
Nothing? Would I see a difference either at compile-time or at execute
time if I left it out?
-Rasmus
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Rasmus Lerdorf wrote:
So how about something like this:
generator function f() {
echo Hello World;
}
generator function f() {
return 1;
}
generator function f($arg) {
if(f!$arg) yield 1;
else if($arg0) return 1;
else return;
}
What does the generator keyword mean in each of these cases? Anything?
Nothing? Would I see a difference either at compile-time or at execute
time if I left it out?
Well the same could be said of some of the error messages generated by 'static'
if E_STRICT is enabled. However an error on the first two saying 'no yield point
found' would not be unreasonable. In reality a lot of this should simply be
handled by a good IDE leaving the runtime code as clean as possible?
For the third one ... I'm still waiting for some clarification on how yield is
SUPPOSED to work anyway? If you are using a 'generator' to return a sequence of
data elements, then just what does happen between each call to the generator ...
DOES the generator get called for each value until something flags there are no
more? I still don't see the advantage of restructuring how things are called to
get around a problem which can just as easily be solved by proper design in the
first place.
--
Lester Caine - G8HFL
-
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Mon, Aug 20, 2012 at 1:56 PM, Ángel González keis...@gmail.com wrote:
On 20/08/12 02:01, Rasmus Lerdorf wrote:
I would still like to understand what this generator keyword would
actually do. I don't see how it would work. Would a function marked
generator somehow not be allowed to return normally or to finish and not
return anything? How could this be enforced? I am completely against any
keyword that is essentially documentation-only.
-Rasmus
Given that such function could return several times, seems a different
enough function type to have its keyword.
The method signature defines the API, so the caller knows what to use.
Can we agree on that? In this case it makes absolutely no difference
to the caller whether the function is implemented using a generator,
or whether it returns a custom Iterator object. The generator
keyword wouldn't document the API, it would document an
implementation-detail.
What would *actually* make sense here are return value typehints. E.g.
one could have something like `public Iterator getIterator() { ... }`.
This would provide the caller with information on what the function
returns, but would leave out the implementation detail that the
Iterator was implemented using a generator. Or, if the generator
implementation is actually important (because it is used as a
coroutine) one could also explicitly typehint against it: `public
Generator getCoroutine() { ... }`.
But return-value type hints are not directly related to generators.
They are a more general concept. If that's what all of you want, then
I'd recommend opening up a new thread for it.
You could not decorate it and rely instead on the presence of the yield
keyword, but parsers will thank knowing about it from the start rather
than realising at mid-parsing that the function is a completely
different beast.
No, parsers don't care about this. It's trivial to detect in both cases.
Nikita
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Mon, Aug 20, 2012 at 3:48 PM, Rasmus Lerdorf ras...@lerdorf.com wrote:
On 08/20/2012 07:56 AM, Ángel González wrote:
On 20/08/12 02:01, Rasmus Lerdorf wrote:
I would still like to understand what this generator keyword would
actually do. I don't see how it would work. Would a function marked
generator somehow not be allowed to return normally or to finish and not
return anything? How could this be enforced? I am completely against any
keyword that is essentially documentation-only.
-Rasmus
Given that such function could return several times, seems a different
enough function type to have its keyword.
You could not decorate it and rely instead on the presence of the yield
keyword, but parsers will thank knowing about it from the start rather
than realising at mid-parsing that the function is a completely
different beast.
So how about something like this:
generator function f() {
echo Hello World;
}
generator function f() {
return 1;
}
generator function f($arg) {
if(f!$arg) yield 1;
else if($arg0) return 1;
else return;
}
What does the generator keyword mean in each of these cases? Anything?
Nothing? Would I see a difference either at compile-time or at execute
time if I left it out?
-Rasmus
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
I still don't think that we shall separate it into two different keywords,
and even if so - we shall still reserve the ability to use return.
Think of this case:
function keywordsInFile($file, $keyword) {
if ( ($han = fopen($file, 'r') === FALSE ) {
return; // -- this is better than starting to wrap the
function with if-else
}
while(!feof(...)) {
foreach ($keywords as $k) {
if ( strpos($k, $line) !== FALSE ) { yield $k; }
}
}
}
Though it's not the best example and I can think about more complicated -
it can reveal my point - I wish to use the return keyword in order to
stop the current function run. In many complicated sites you got many
if-else cases in one function that when the statement evaluates to
true/false you shall return from the function.
Structure like
if ( ... ) {
if ( ! ... ) {
if ( ) {
if ( ... ) {
yield $v;
}
}
}
}
is ugly and make the program harder to read than
if ( ! ... ) return;
if ( ... ) return;
if (! ... ) return;
if ( ! ... ) return;
yield $k;
Re: [PHP-DEV] [RFC] Generators
Rasmus Lerdorf:
So how about something like this:
generator function f() {
echo Hello World;
}
generator function f() {
return 1;
}
generator function f($arg) {
if(f!$arg) yield 1;
else if($arg0) return 1;
else return;
}
What does the generator keyword mean in each of these cases? Anything?
Nothing? Would I see a difference either at compile-time or at execute
time if I left it out?
An excellent argument to use yield where it is needed, instead
of using generator at the top.
Wietse
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] [RFC] Generators
-Original Message-
From: Nikita Popov [mailto:nikita@gmail.com]
Sent: 20 August 2012 14:12
To: Ángel González
Cc: Rasmus Lerdorf; Stas Malyshev; Derick Rethans; PHP internals
Subject: Re: [PHP-DEV] [RFC] Generators
On Mon, Aug 20, 2012 at 1:56 PM, Ángel González
keis...@gmail.com wrote:
On 20/08/12 02:01, Rasmus Lerdorf wrote:
I would still like to understand what this generator keyword
would
actually do. I don't see how it would work. Would a
function marked
generator somehow not be allowed to return normally or to
finish and
not return anything? How could this be enforced? I am completely
against any keyword that is essentially documentation-only.
-Rasmus
Given that such function could return several times, seems a
different enough function type to have its keyword.
The method signature defines the API, so the caller knows what to
use.
Can we agree on that? In this case it makes absolutely no
difference to the caller whether the function is implemented
using a generator, or whether it returns a custom Iterator
object. The generator
keyword wouldn't document the API, it would document an
implementation-detail.
What would *actually* make sense here are return value typehints.
E.g.
one could have something like `public Iterator getIterator() { ...
}`.
This would provide the caller with information on what the
function returns, but would leave out the implementation
detail that the Iterator was implemented using a generator.
Or, if the generator implementation is actually important
(because it is used as a
coroutine) one could also explicitly typehint against it:
`public Generator getCoroutine() { ... }`.
Yes. Quick example...
interface A
{
function gen();
}
class AImplementation implements A
{
function gen()
{
for($i = 0; $i 10; ++$i)
yield $i;
}
}
class ADecorator implements A
{
private $a;
function __construct(A $a)
{
$this-a = $a;
}
function gen()
{
return $this-a-gen();
}
}
$a = new ADecorator(new AImplementation());
foreach($a-gen() as $v)
echo $v, \n;
Obviously, AImplementation::gen() is a generator, but
ADecorator::gen() isn't, but would want both to adhere to the same
interface.
So only return type hinting makes sense.
But return-value type hints are not directly related to generators.
They are a more general concept. If that's what all of you
want, then I'd recommend opening up a new thread for it.
You could not decorate it and rely instead on the presence of the
yield keyword, but parsers will thank knowing about it from
the start
rather than realising at mid-parsing that the function is a
completely
different beast.
No, parsers don't care about this. It's trivial to detect in
both cases.
Nikita
--
PHP Internals - PHP Runtime Development Mailing List To
unsubscribe, visit: http://www.php.net/unsub.php
Jared
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Hi!
generator function f() {
echo Hello World;
}
even more interesting,
$a = generator function() {
echo Hello World;
}
or even:
function foo() {
return generator function() {
echo Hello World;
}
}
$a = foo();
--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Hi!
What would *actually* make sense here are return value typehints. E.g.
one could have something like `public Iterator getIterator() { ... }`.
And again we're back to inventing syntax to do what documentation should
be doing.
--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 20/08/12 19:46, Stas Malyshev wrote:
Hi!
What would *actually* make sense here are return value typehints. E.g.
one could have something like `public Iterator getIterator() { ... }`.
And again we're back to inventing syntax to do what documentation should
be doing.
For the large part, but consider type hints mean that if you return a
completely wrong type, you can warn the programmer.
--
Andrew Faulds
http://ajf.me/
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 20/08/12 15:12, Nikita Popov wrote: You could not decorate it and rely instead on the presence of the yield keyword, but parsers will thank knowing about it from the start rather than realising at mid-parsing that the function is a completely different beast. No, parsers don't care about this. It's trivial to detect in both cases. Nikita Yes, it's trivial to detect, but how much work is for it to do so? Suppose the parser reads php code and outputs machine instructions. When it encouters a function(), it adds the function prologue, goes reserving stack space for the variables it encounters, and so on. Then you find a yield keyword. Options: - Discard the generated code and go back to the function begin, using this time an storage into an object instead of the stack. - Copy all the state to an object and return it. - Add an initial pass checking which functions are generators. PHP being a dynamic language, it may be able to move the function variables to a class. But it's not trivial for any parser. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 18/08/12 13:21, Derick Rethans wrote:
On Sat, 11 Aug 2012, Nikita Popov wrote:
Hi internals!
I think there already was a lot of discussion on the generators, so
it's time to move to the next step. I'd like to vote on the feature in
two weeks, so this the announce[ment] on internals@, by the author,
with the intention of voting on it.
https://wiki.php.net/rfc/generators
If you have any further feedback, now would be a good time to raise
it. If there is something you previously posted, but which I didn't
yet address, please let me know. There were around 150 mails and I
sure missed some of them.
I've some comments how that I've read the RFC:
Recognition of generator functions
1. Any function which contains a yield statement is automatically a
generator function.
2. The initial implementation required that generator functions are
marked with an asterix modifier (function*). This method has the
advantage that generators are more explicit and also allows for
yield-less coroutines.
The automatic detection was chosen over the asterix modifier for the
following reasons:
I am against this. This is even more magic in PHP. Is it really that
difficult to have to mark the function with a different keyword, such as
generator:
You know, this conversation feels like it's going in circles. I thought
we already agreed there wouldn't be one.
generator function getLinesFromFile($fileName) {
if (!$fileHandle = fopen($fileName, 'r')) {
return;
}
There is an existing generator implementation in HipHop PHP, which
uses automatic-detection. Using the asterix modifier would break
compatibility.
This should not be a concern, sure, it's annoying for the hiphop
developers but they chose to copy and then *chance* the PHP language for
their own effect.
yield: Yields the value null with an auto-incrementing integer key.
What is the usecase for this?
cheers,
Derick
--
Andrew Faulds
http://ajf.me/
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
I think this doesn't bring anything further to talk about how to
implement this in the parser.
After reading the thread again, I would like to say that
- either this should be implemented as currently defined
- or it is implemented with the generator-keyword. But then it isn't a
function anymore, because that's is ridiculous.
In that case it should look like this:
generator g() {
yield;
return;
}
2012/8/20 Ángel González keis...@gmail.com:
On 20/08/12 15:12, Nikita Popov wrote:
You could not decorate it and rely instead on the presence of the yield
keyword, but parsers will thank knowing about it from the start rather
than realising at mid-parsing that the function is a completely
different beast.
No, parsers don't care about this. It's trivial to detect in both cases.
Nikita
Yes, it's trivial to detect, but how much work is for it to do so?
Suppose the parser reads php code and outputs machine instructions. When
it encouters a function(), it adds the function prologue, goes reserving
stack
space for the variables it encounters, and so on. Then you find a yield
keyword.
Options:
- Discard the generated code and go back to the function begin, using
this time
an storage into an object instead of the stack.
- Copy all the state to an object and return it.
- Add an initial pass checking which functions are generators.
PHP being a dynamic language, it may be able to move the function variables
to a class. But it's not trivial for any parser.
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
--
Freundliche Grüße
Alex Aulbach
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Mon, Aug 20, 2012 at 10:07 PM, Ángel González keis...@gmail.com wrote: On 20/08/12 15:12, Nikita Popov wrote: You could not decorate it and rely instead on the presence of the yield keyword, but parsers will thank knowing about it from the start rather than realising at mid-parsing that the function is a completely different beast. No, parsers don't care about this. It's trivial to detect in both cases. Nikita Yes, it's trivial to detect, but how much work is for it to do so? Suppose the parser reads php code and outputs machine instructions. When it encouters a function(), it adds the function prologue, goes reserving stack space for the variables it encounters, and so on. Then you find a yield keyword. Options: - Discard the generated code and go back to the function begin, using this time an storage into an object instead of the stack. - Copy all the state to an object and return it. - Add an initial pass checking which functions are generators. PHP being a dynamic language, it may be able to move the function variables to a class. But it's not trivial for any parser. Sorry, I don't understand what we are arguing about here. I implemented this. I did the parser changes for this feature. And I told you that they were simple either way (both with keyword, as in the initial implementation, and without keyword, as in the current implementation). For 3rd parties (like IDEs) it is even simpler because their parsers are AST-based (unlike PHP's own parser), so they shouldn't care at all. Honestly, I stopped understanding what this whole discussion is about around ~100 mails ago. Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: Re: [PHP-DEV] [RFC] Generators
On 2012-08-21 01:10, Lester Caine wrote: For the third one ... I'm still waiting for some clarification on how yield is SUPPOSED to work anyway? If you are using a 'generator' to return a sequence of data elements, then just what does happen between each call to the generator ... DOES the generator get called for each value until something flags there are no more? I still don't see the advantage of restructuring how things are called to get around a problem which can just as easily be solved by proper design in the first place. Have you even read the RFC yet? Calling a generator function returns an instance of a Generator object. That object has a method interface - described in the RFC. That interface is accessible to the foreach() construct so the loop can iterate over the successive values returned by the object. If by proper design you mean your own personal preference for using callbacks, that's also discussed in the RFC. You were told this last month: On Sat, Jul 28, 2012 as 05:34 AM, Nikita Popov nikita@gmail.com wrote On Wed, Jul 25, 2012 at 10:36 PM, Lester Caine les...@lsces.co.uk wrote: But WHY would you not simply put the function that is handling the returned data in place of the yield? Why do you want to keep exiting and re-entering the 'do loop' when the data can simply be forwarded direct to a function to handle it? This question has come up a few times now, i.e. why one can't just use callbacks. So I added a section about this, explaining it with a few examples: https://wiki.php.net/rfc/generators#why_not_just_use_callback_functions Hope it helps, Nikita But let's compare the two side-by-side. So there is a producer that has or generates the successive elements of a sequence, and a consumer that does something with each element in turn. One can make a choice about which of the two - producer or consumer - gets to drive the process. In one (basically the Observer pattern), the producer drives, and the consumer gets to observe by supplying a callback that the producer calls for each new element. Since the producer has no idea about the internal workings of the consumer, the consumer has to ensure that it can properly resume the next time it is called. In the other (basically the Iterator pattern), the consumer drives, calling the producer each time a new element is required. Since the consumer has no idea about the internal workings of the producer, the producer has to ensure that it can properly resume the next time it is called. There are a couple of differences between using an observer and using an iterator. The main difference is because at least one of the producer or consumer needs to maintain its own state between iterations. For the producer-driven observer approach, it is the consumer's job to maintain state. - the producer's state is safe on the call stack. For the consumer-driven iterator approach, it is the producer's job to maintain state - the consumer's state is safe on the call stack. In the observer approach, writing the state-maintenance code falls to whoever is writing the consumer; the producer is unable to help with that at all because it's different for each consumer. In the iterator approach, writing the state-maintenance code falls to whoever is writing the producer; that code is the same regardless of the consumer. As this proposal and existing implementations show, it is possible, given the kernel behaviour of the iteration, to mechanically generate all the state-handling and other boilerplate necessary to produce a Generator object. That's the main difference. The other difference is that in the case of the iterator, in part because the code generation can be mechanised, the iteration mechanism can be done in such a way that it fits in with the iteration mechanisms that PHP already has built into the language: the foreach() statement doesn't care whether it's iterating over an array or an iterator or a generator. Either could be supplied at runtime. In contrast, to achieve the analogous effect with an observer mechanism would involve (among other things) searching for occurrences of control flow statements that reference the sequence, excising those statements and using them as the kernel for a callback method to be supplied to the sequence producer ... but only if the sequence in question wants all that done in the first place. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Hi! I am against this. This is even more magic in PHP. Is it really that difficult to have to mark the function with a different keyword, such as generator: You have a point here, but public static final generator function foo() sounds a bit long-winded to me... Also, we'd have then to decide which function can be marked generator and which can't (e.g., interface probably can't, abstract probably can't, anonymous probably can, etc.) which adds more complexity. Also, I think that people that complain about having to scan through huge functions to see if they're generators or not, forget one thing: documentation. Yes, there is a way to make the purpose of the function understandable to a human without having him to do computer's work. That's documentation. Undocumented code is broken code. Broken code is not a good example when we're talking about right design. This should not be a concern, sure, it's annoying for the hiphop developers but they chose to copy and then *chance* the PHP language for their own effect. Here I tend to agree with you - we should base on what's right for PHP, not what HipHop or any other implementation is doing. If we can make their lives easier - fine, but we don't have to be bound by their decisions. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 08/19/2012 07:07 PM, Stas Malyshev wrote: Hi! I am against this. This is even more magic in PHP. Is it really that difficult to have to mark the function with a different keyword, such as generator: You have a point here, but public static final generator function foo() sounds a bit long-winded to me... Also, we'd have then to decide which function can be marked generator and which can't (e.g., interface probably can't, abstract probably can't, anonymous probably can, etc.) which adds more complexity. Also, I think that people that complain about having to scan through huge functions to see if they're generators or not, forget one thing: documentation. Yes, there is a way to make the purpose of the function understandable to a human without having him to do computer's work. That's documentation. Undocumented code is broken code. Broken code is not a good example when we're talking about right design. I would still like to understand what this generator keyword would actually do. I don't see how it would work. Would a function marked generator somehow not be allowed to return normally or to finish and not return anything? How could this be enforced? I am completely against any keyword that is essentially documentation-only. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] [RFC] Generators
-Original Message-
From: Stas Malyshev [mailto:smalys...@sugarcrm.com]
Sent: 20 August 2012 00:08
To: Derick Rethans
Cc: Nikita Popov; PHP internals
Subject: Re: [PHP-DEV] [RFC] Generators
Hi!
I am against this. This is even more magic in PHP. Is it
really that
difficult to have to mark the function with a different
keyword, such
as
generator:
You have a point here, but public static final generator
function foo() sounds a bit long-winded to me... Also, we'd
have then to decide which function can be marked generator
and which can't (e.g., interface probably can't, abstract
probably can't, anonymous probably can, etc.) which adds more
complexity.
That'd be return type hinting would it not?
Given that generator functions currently return an instance of class
Generator.
Also anonymous generators would be quite strange
$f = Generator function() { yield 'a'; };
Jared
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Tue, Aug 14, 2012 at 7:59 PM, Aaron Holmes aa...@aaronholmes.net wrote: Thanks for clarifying. It makes sense now, considering foreach's behavior and the generators statefulness allowing what otherwise seems inconsistent. However, might it make sense to no-op instead of erroring? If generators allow rewind(), it would be unexpected to receive an error for calling it. A no-op is the current behavior. The issue with that is that you could accidentally reuse an already depleted (or partially depleted) generator. I.e. you run it through one loop and then through another one. With a no-op rewind it would be hard to figure out what the issue is. The error makes clear that you tried to iterate an already iterated generator. Furthermore, if you indeed *want* to partially traverse a generator in several loops, you can simply wrap it in a NoRewindIterator. This has the additional benefit of making it more clear what you want to do. Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sat, 11 Aug 2012, Nikita Popov wrote:
Hi internals!
I think there already was a lot of discussion on the generators, so
it's time to move to the next step. I'd like to vote on the feature in
two weeks, so this the announce[ment] on internals@, by the author,
with the intention of voting on it.
https://wiki.php.net/rfc/generators
If you have any further feedback, now would be a good time to raise
it. If there is something you previously posted, but which I didn't
yet address, please let me know. There were around 150 mails and I
sure missed some of them.
I've some comments how that I've read the RFC:
Recognition of generator functions
1. Any function which contains a yield statement is automatically a
generator function.
2. The initial implementation required that generator functions are
marked with an asterix modifier (function*). This method has the
advantage that generators are more explicit and also allows for
yield-less coroutines.
The automatic detection was chosen over the asterix modifier for the
following reasons:
I am against this. This is even more magic in PHP. Is it really that
difficult to have to mark the function with a different keyword, such as
generator:
generator function getLinesFromFile($fileName) {
if (!$fileHandle = fopen($fileName, 'r')) {
return;
}
There is an existing generator implementation in HipHop PHP, which
uses automatic-detection. Using the asterix modifier would break
compatibility.
This should not be a concern, sure, it's annoying for the hiphop
developers but they chose to copy and then *chance* the PHP language for
their own effect.
yield: Yields the value null with an auto-incrementing integer key.
What is the usecase for this?
cheers,
Derick
--
http://derickrethans.nl | http://xdebug.org
Like Xdebug? Consider a donation: http://xdebug.org/donate.php
twitter: @derickr and @xdebug
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sat, Aug 18, 2012 at 2:21 PM, Derick Rethans der...@php.net wrote: I've some comments how that I've read the RFC: Recognition of generator functions 1. Any function which contains a yield statement is automatically a generator function. 2. The initial implementation required that generator functions are marked with an asterix modifier (function*). This method has the advantage that generators are more explicit and also allows for yield-less coroutines. The automatic detection was chosen over the asterix modifier for the following reasons: I am against this. This is even more magic in PHP. Is it really that difficult to have to mark the function with a different keyword, such as generator: There was already a rather long discussion on this topic. Most of it is noise, but I filtered out a few mails from Sara and Rasmus as they probably are of most interest: http://markmail.org/message/xzhdhbjozb4yrhh3 http://markmail.org/message/ryhygtimpd7q2nok http://markmail.org/message/32fklwqykpk56iph http://markmail.org/message/w2kbh7psplnmctcr yield: Yields the value null with an auto-incrementing integer key. What is the usecase for this? Use case is `$data = yield;`. In that case you don't care about what you yield (so just the default value and key are yielded), you are only interested in what you get sent back. Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sat, 11 Aug 2012, Nikita Popov wrote:
Hi internals!
I think there already was a lot of discussion on the generators, so
it's time to move to the next step. I'd like to vote on the feature in
two weeks, so this the announce[ment] on internals@, by the author,
with the intention of voting on it.
https://wiki.php.net/rfc/generators
If you have any further feedback, now would be a good time to raise
it. If there is something you previously posted, but which I didn't
yet address, please let me know. There were around 150 mails and I
sure missed some of them.
I've some comments how that I've read the RFC:
Recognition of generator functions
1. Any function which contains a yield statement is automatically a
generator function.
2. The initial implementation required that generator functions are
marked with an asterix modifier (function*). This method has the
advantage that generators are more explicit and also allows for
yield-less coroutines.
The automatic detection was chosen over the asterix modifier for the
following reasons:
I am against this. This is even more magic in PHP. Is it really that
difficult to have to mark the function with a different keyword, such as
generator:
Since this is yet another area where 'one does not have to use it if one
does not want to' ... FLAGGING to the other users that a function is a
'special one' rather than just a normal function with a generator function
seems to me just a necessity? HAVING to work through a functions code
simply to see if it contains a 'yeild' so as to understand that it's not
going to give a normal return seems insane? Alright the function can be
properly documented with a docblock, but if it isn't ...
generator function getLinesFromFile($fileName) {
if (!$fileHandle = fopen($fileName, 'r')) {
return;
}
There is an existing generator implementation in HipHop PHP, which
uses automatic-detection. Using the asterix modifier would break
compatibility.
This should not be a concern, sure, it's annoying for the hiphop
developers but they chose to copy and then *chance* the PHP language for
their own effect.
yield: Yields the value null with an auto-incrementing integer key.
What is the usecase for this?
I can see some interesting porting problems with this ... one of the stock
changes when moving from a MySQL only setup to support other databases is
to remove the auto-increment magic, and replace it with proper sequence
code prior to inserting. I can see translating a 'MySQL' geared generator
into a more general one as being impossible if some tricks like this are
used.
(Out in an exhibition hall this weekend so not able to follow properly)
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 08/18/2012 10:12 AM, les...@lsces.co.uk wrote: Since this is yet another area where 'one does not have to use it if one does not want to' ... FLAGGING to the other users that a function is a 'special one' rather than just a normal function with a generator function seems to me just a necessity? HAVING to work through a functions code simply to see if it contains a 'yeild' so as to understand that it's not going to give a normal return seems insane? Alright the function can be properly documented with a docblock, but if it isn't ... I don't see how that is any different from having to look through a function to see if it is a void function or if it returns something, and if so, what it returns. Or checking to see if it outputs something. PHP is very dynamic. A function can be both void and non-void at the same time. It can return mixed types and do pretty much anything you can imagine. I don't really see what the generator keyword achieves. What happens if I have a function that I mark as a generator but then I don't put a yield in it? Is that a compile-level error? That might work, but then what if I put a yield in it but the logic of the function never gets to it: if(cond) return [1,2,3]; else if(other_cond) yield $val; else return; There is no way to know at compile-time whether the yield will ever be reached. To me it seems odd to have a generator keyword that cannot be enforced and doesn't mean anything other than stating a developer's intent. Whether the developer actually carried through and implemented the intended generator can only be verified by reading the code. This smells much more like a source comment to me. If we were going to go with a generator keyword, then I think the feature needs to be completely re-thought. I would suggest dropping the yield keyword entirely and having return be equivalent to a yield if the function is marked as a generator. Then the keyword actually does something. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sat, Aug 18, 2012 at 4:12 PM, les...@lsces.co.uk wrote:
Since this is yet another area where 'one does not have to use it if one
does not want to' ... FLAGGING to the other users that a function is a
'special one' rather than just a normal function with a generator function
seems to me just a necessity? HAVING to work through a functions code
simply to see if it contains a 'yeild' so as to understand that it's not
going to give a normal return seems insane? Alright the function can be
properly documented with a docblock, but if it isn't ...
I don't understand this argument. Generator functions are transparent
to the user. You use a generator function just like you would use a
function that returns an array. From a user point of view it does not
matter whether getLinesFromFile() is just a function returning an
array, or whether it is a generator (or maybe even returns a
hand-implemented iterator). It's just all the same. The fact that the
function uses `yield` internally is just an implementation detail, not
something that has to be part of the public API. Actually you can swap
between an array and generator implementation just by replacing one
line in the function body (yield = $array[])...
generator function getLinesFromFile($fileName) {
if (!$fileHandle = fopen($fileName, 'r')) {
return;
}
There is an existing generator implementation in HipHop PHP, which
uses automatic-detection. Using the asterix modifier would break
compatibility.
This should not be a concern, sure, it's annoying for the hiphop
developers but they chose to copy and then *chance* the PHP language for
their own effect.
yield: Yields the value null with an auto-incrementing integer key.
What is the usecase for this?
I can see some interesting porting problems with this ... one of the stock
changes when moving from a MySQL only setup to support other databases is
to remove the auto-increment magic, and replace it with proper sequence
code prior to inserting. I can see translating a 'MySQL' geared generator
into a more general one as being impossible if some tricks like this are
used.
Generators are supposed to act very similar to array-returning
functions, so they also have similar key semantics. If you do
$result[] = $foo; that will behave the same as doing yield $foo;. Both
will use an auto-incrementing key.
Nikita
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Nikita Popov wrote: I don't understand this argument. Generator functions are transparent to the user. You use a generator function just like you would use a function that returns an array. From a user point of view it does not matter whether getLinesFromFile() is just a function returning an array, or whether it is a generator (or maybe even returns a hand-implemented iterator). It's just all the same. The fact that the function uses `yield` internally is just an implementation detail, not something that has to be part of the public API. Actually you can swap between an array and generator implementation just by replacing one line in the function body (yield = $array[])... Then I am now totally confused ... I was under the impression that the IDEA was that EACH call to a generator would return the next value? So you do not get an array built. Just as one would with SUSPEND in an SQL query process. The only value available is the current one, and one processes that. So that the user of a generator needs to handle the results sequentially rather than simply accessing the array of results. There does not need to be an array of results, only the processed output of each cycle of the generator? This of cause may still be confused thinking, since I would never be working in the way that this stuff is supposed to simplify. I WOULD be handling each element as I read it and building the details required from each cycle of the process. getLinesFromFile() would only ever return an array if that is what was needed to build the resulting page. It would normally be processed into a database, and later the required information selected from that. But that is a specialist iterator process just built with simple PHP. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
2012/8/13 Anthony Ferrara ircmax...@gmail.com: It's absolutely something that takes getting used to. But it's also quite intuitive once you think about it. You're not returning back to the parent scope (exiting your scope), you're yielding a value to the parent scope, expecting to continue on later (think of it as a stop light where you let the other code run for a while until you go to the next one). I think for people, which are - like me - more used with databases, the cursor-concept is a good comparison. Perhaps this could help a little bit to explain it... -- Alex Aulbach -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
2012/8/13 Nikita Popov nikita@gmail.com: * After first yield: Recoverable fatal error So this would allow you to call -rewind() after creating the generator, but will throw an error if you try to do so later. Hm. I think this looks like a workaround over conceptual problems. The (in my eyes) right behavior would be to call a new instance of the generator if you rewind and destroy the current (= replace current generator with a new). I would prefer to say this is not an real iterator; if you want to make it an real iterator, then implement it yourself. Otherwise it's like Ok, we have here a new type of function which looks enough like a function to be a function, which implements an iterator which looks enough like an iterator to be an iterator.. :) I also recommend to say, that generators aren't a good idea if used on resources, which can't be shared. For example: the file-read-example doesn't lock. To implement locking new functions or a new class is needed - hiding complexity. Generators are (in my opinion) really nice for things which are already in the memory. Not more or less. -- Alex Aulbach -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
If I may chime in, not all iterators support rewind, at least not as outlined. There exists also the NoRewindIterator which you can just use to decorate an iterator out of a generator to not make it do a fatal. However as far as a generator is concerned, it probably should just behave like NoRewindIterator already so not give any errors on rewind. Because you can not rewind what you can not rewind. No need to error, no need to do something. The function is just there because of the interface, but it must not do anything. If different behavior is needed it could be decorated, but right now I can not imagine why rewind would be called (apart from using in foreach() for the first time which implies a rewind and that one is already like I suggest.) My 2 cents. -- hakre - Ursprüngliche Message - Von: Nikita Popov nikita@gmail.com An: Brian Moon br...@moonspot.net CC: PHP internals internals@lists.php.net Gesendet: 18:09 Montag, 13.August 2012 Betreff: Re: [PHP-DEV] [RFC] Generators On Sun, Aug 12, 2012 at 10:08 PM, Brian Moon br...@moonspot.net wrote: Also, not allowing rewinding is unintuitive for something that is an iterator in PHP. If I can foreach() it and I can call next() on it, I expect to be able to reset() it as well. IMO, you would need to issue a FATAL PHP error if that simply is not allowed. Or you have to have a second syntax for what to do in that case. At that point, you are implementing Iterator. Currently I'm planning to implement the following behavior for rewind(): * If before first yield: Resume to first yield (this priming behavior is common to all the Iterator methods) * At first yield: No-op * After first yield: Recoverable fatal error So this would allow you to call -rewind() after creating the generator, but will throw an error if you try to do so later. Another thing that I'd like to do is drop the -close() method. It doesn't really make sense to explicitly close generators. Any objects to those two things? Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 8/13/12 9:09 AM, Nikita Popov wrote: On Sun, Aug 12, 2012 at 10:08 PM, Brian Moon br...@moonspot.net wrote: Also, not allowing rewinding is unintuitive for something that is an iterator in PHP. If I can foreach() it and I can call next() on it, I expect to be able to reset() it as well. IMO, you would need to issue a FATAL PHP error if that simply is not allowed. Or you have to have a second syntax for what to do in that case. At that point, you are implementing Iterator. Currently I'm planning to implement the following behavior for rewind(): * If before first yield: Resume to first yield (this priming behavior is common to all the Iterator methods) * At first yield: No-op * After first yield: Recoverable fatal error So this would allow you to call -rewind() after creating the generator, but will throw an error if you try to do so later. My perspective is that generators are intended to generate and yield a result, not so much to iterate over a known set of results. Thus, the results not really being known, there is nothing to rewind to. Rewinding would also presumably require more state control, or keeping track of previous results, and defeat the memory advantages of generators. Perhaps this is erroneous? That said, rewind() should behave consistently. I don't feel it makes sense to have rewind() succeed at one point, and fail at another. It would only cause confusion when not familiar with the behavior. Either allow it, or don't. Not both. Thanks, Aaron Holmes -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Hi! That said, rewind() should behave consistently. I don't feel it makes sense to have rewind() succeed at one point, and fail at another. It would only cause confusion when not familiar with the behavior. Either allow it, or don't. Not both. It does, since foreach uses rewind. So first rewind should succeed if you want iterators be usable in foreach. OTOH, on something like DB result set, next rewind does not make any sense, if you have non-seekable cursor, since the results consumed are gone and there's no way to get them back (you could rerun the query, but it might have side effects and nobody guarantees you'd get the same result anyway). So it makes sense for the generator to succeed on first rewind but fail on next ones. Note that generators by nature are stateful objects, so it is not unexpected that they would produce different result in different states. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 8/14/12 10:36 AM, Stas Malyshev wrote: Hi! That said, rewind() should behave consistently. I don't feel it makes sense to have rewind() succeed at one point, and fail at another. It would only cause confusion when not familiar with the behavior. Either allow it, or don't. Not both. It does, since foreach uses rewind. So first rewind should succeed if you want iterators be usable in foreach. OTOH, on something like DB result set, next rewind does not make any sense, if you have non-seekable cursor, since the results consumed are gone and there's no way to get them back (you could rerun the query, but it might have side effects and nobody guarantees you'd get the same result anyway). So it makes sense for the generator to succeed on first rewind but fail on next ones. Note that generators by nature are stateful objects, so it is not unexpected that they would produce different result in different states. Thanks for clarifying. It makes sense now, considering foreach's behavior and the generators statefulness allowing what otherwise seems inconsistent. However, might it make sense to no-op instead of erroring? If generators allow rewind(), it would be unexpected to receive an error for calling it. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP-DEV] [RFC] Generators
-Original Message-
From: Anthony Ferrara [mailto:ircmax...@gmail.com]
Sent: 13 August 2012 03:49
To: Brian Moon
Cc: Nikita Popov; PHP internals
Subject: Re: [PHP-DEV] [RFC] Generators
Brian,
On Sun, Aug 12, 2012 at 4:08 PM, Brian Moon
br...@moonspot.net wrote:
Hi Nikita,
I admit, I have ignored these threads as there was no RFC.
So, some of
this may have been covered.
There was an RFC in those posts... It was just being iterated over.
Do you have a good example usage other than a file? I don't find
fopen/fgets/fclose all that complicated. What are the other
valid use
cases for such a thing?
Here's a quick set of examples:
http://blog.ircmaxell.com/2012/07/what-generators-can-do-for-you.html
Also, not allowing rewinding is unintuitive for something
that is an
iterator in PHP. If I can foreach() it and I can call
next() on it, I
expect to be able to reset() it as well. IMO, you would
need to issue
a FATAL PHP error if that simply is not allowed. Or you
have to have a
second syntax for what to do in that case. At that point, you are
implementing Iterator.
I partially agree. rewinding the generator might be possible,
but it's hard to tell in those cases. It's hard to tell if
resetting it is even possible from the code level. So I'm
pretty much OK with living with that restriction for the time
being...
While I am glad that PHP has borrowed syntax from many languages,
I
find the yield keyword to be very WTF when I first saw it.
It is also
poorly explained in your RFC. You use terms like sending and
receiving. That does not say returns from function
execution to me.
I had to basically figure it out from the code examples.
It's absolutely something that takes getting used to. But
it's also quite intuitive once you think about it. You're not
returning back to the parent scope (exiting your scope),
you're yielding a value to the parent scope, expecting to
continue on later (think of it as a stop light where you let
the other code run for a while until you go to the next one).
Lastly, because you cite needing this for sending data to
PHPUnit, I
think this is a user land problem and not a core problem.
In about 5
minutes I implemented a reusable Generator object that does
exactly what you need.
http://pastebin.com/V336rEpR At least, it does what your
examples show
you need. Again, file IO is very easy and perhaps that example
does
not explain your true need very well.
Well, there's one thing that should be made clear. There's
nothing (and I mean that) that generators provide that you
can't do already. You can build iterators that do exactly the
same thing.
Not true. Iterator::current() cannot return references, but generators
can yield them.
Eg
function map(array $row, array $order)
{
foreach($order as $index)
yield $row[$index];
}
foreach(map($row, [1,3,5,7,9]) as $ref)
Anthony
Jared
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sun, Aug 12, 2012 at 10:08 PM, Brian Moon br...@moonspot.net wrote: Also, not allowing rewinding is unintuitive for something that is an iterator in PHP. If I can foreach() it and I can call next() on it, I expect to be able to reset() it as well. IMO, you would need to issue a FATAL PHP error if that simply is not allowed. Or you have to have a second syntax for what to do in that case. At that point, you are implementing Iterator. Currently I'm planning to implement the following behavior for rewind(): * If before first yield: Resume to first yield (this priming behavior is common to all the Iterator methods) * At first yield: No-op * After first yield: Recoverable fatal error So this would allow you to call -rewind() after creating the generator, but will throw an error if you try to do so later. Another thing that I'd like to do is drop the -close() method. It doesn't really make sense to explicitly close generators. Any objects to those two things? Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Hi Nikita, I admit, I have ignored these threads as there was no RFC. So, some of this may have been covered. Do you have a good example usage other than a file? I don't find fopen/fgets/fclose all that complicated. What are the other valid use cases for such a thing? Also, not allowing rewinding is unintuitive for something that is an iterator in PHP. If I can foreach() it and I can call next() on it, I expect to be able to reset() it as well. IMO, you would need to issue a FATAL PHP error if that simply is not allowed. Or you have to have a second syntax for what to do in that case. At that point, you are implementing Iterator. While I am glad that PHP has borrowed syntax from many languages, I find the yield keyword to be very WTF when I first saw it. It is also poorly explained in your RFC. You use terms like sending and receiving. That does not say returns from function execution to me. I had to basically figure it out from the code examples. Lastly, because you cite needing this for sending data to PHPUnit, I think this is a user land problem and not a core problem. In about 5 minutes I implemented a reusable Generator object that does exactly what you need. http://pastebin.com/V336rEpR At least, it does what your examples show you need. Again, file IO is very easy and perhaps that example does not explain your true need very well. Brian brianlm...@php.net http://brian.moonspot.net/ On 8/11/12 8:42 AM, Nikita Popov wrote: Hi internals! I think there already was a lot of discussion on the generators, so it's time to move to the next step. I'd like to vote on the feature in two weeks, so this the announce[ment] on internals@, by the author, with the intention of voting on it. https://wiki.php.net/rfc/generators If you have any further feedback, now would be a good time to raise it. If there is something you previously posted, but which I didn't yet address, please let me know. There were around 150 mails and I sure missed some of them. Thanks, Nikita -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon br...@moonspot.net wrote:
Do you have a good example usage other than a file? I don't find
fopen/fgets/fclose all that complicated. What are the other valid use cases
for such a thing?
One fabulous use case is creating an iterator for a Binary Search
Tree. A post order done without generators looks like:
https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php.
An iterator using a generator looks something like:
public function getIterator() {
if ($this-left) yield* $this-left;
yield $this-value;
if ($this-right) yield* $this-right;
}
This is 5 lines. The fully commented version of the post-order
iterator previously mentioned is 106 lines of code and is considerably
harder to understand.
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 13 באוג 2012, at 01:33, Levi Morrison morrison.l...@gmail.com wrote:
On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon br...@moonspot.net wrote:
Do you have a good example usage other than a file? I don't find
fopen/fgets/fclose all that complicated. What are the other valid use cases
for such a thing?
One fabulous use case is creating an iterator for a Binary Search
Tree. A post order done without generators looks like:
https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php.
An iterator using a generator looks something like:
public function getIterator() {
if ($this-left) yield* $this-left;
yield $this-value;
if ($this-right) yield* $this-right;
}
This is 5 lines. The fully commented version of the post-order
iterator previously mentioned is 106 lines of code and is considerably
harder to understand.
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Each Iterator implementation in PHP is longer and require a class. The yield
keyword can reduce the code and make things simpler. If my vote count, though I
can't believe it is, I'd vote for it - clean syntax, make things simpler to
develop after you understand the concept (for those who against adding because
of the learning issue and beginners - we'll have beginners anytime that'll mess
up with the concept, so I don't think that it should restrict us. Put that
aside, we should make the documentation clear and easy as possible in order to
help beginners understand the concept).
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On 8/12/12 5:33 PM, Levi Morrison wrote:
On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon br...@moonspot.net wrote:
Do you have a good example usage other than a file? I don't find
fopen/fgets/fclose all that complicated. What are the other valid use cases for
such a thing?
One fabulous use case is creating an iterator for a Binary Search
Tree. A post order done without generators looks like:
https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php.
An iterator using a generator looks something like:
public function getIterator() {
if ($this-left) yield* $this-left;
yield $this-value;
if ($this-right) yield* $this-right;
}
This is 5 lines. The fully commented version of the post-order
iterator previously mentioned is 106 lines of code and is considerably
harder to understand.
Well, it's 52 lines, not 106. The 5 lines above are not commented nor
are they spaced at all like the ones in the class. In the above example,
what sets $this-right? or $this-left? There has to be more calling
code around this. I don't consider this a very good example.
Brian.
brianlm...@php.net
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sun, Aug 12, 2012 at 5:56 PM, Brian Moon br...@moonspot.net wrote:
On 8/12/12 5:33 PM, Levi Morrison wrote:
On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon br...@moonspot.net wrote:
Do you have a good example usage other than a file? I don't find
fopen/fgets/fclose all that complicated. What are the other valid use cases
for such a thing?
One fabulous use case is creating an iterator for a Binary Search
Tree. A post order done without generators looks like:
https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php.
An iterator using a generator looks something like:
public function getIterator() {
if ($this-left) yield* $this-left;
yield $this-value;
if ($this-right) yield* $this-right;
}
This is 5 lines. The fully commented version of the post-order
iterator previously mentioned is 106 lines of code and is considerably
harder to understand.
Well, it's 52 lines, not 106. The 5 lines above are not commented nor are
they spaced at all like the ones in the class.
No, Brian, it is 106 lines of actual code I would find in the wild.
And sure, I'd have spaced the generator code more like:
public function getIterator() {
if ($this-left) {
yield* $this-left;
}
if ($this-right) {
yield* $this-right;
}
yield $this-value;
}
However, that's 11 lines of code compared to 106.
In the above example, what
sets $this-right? or $this-left?
Iterators don't set $this-right and $this-left; they only traverse the tree.
I don't consider this a very good example.
That's fine. You are entitled to your opinion. However, you didn't
have to figure out the 106 lines of code that is the Post-Order
iterator. That was a NIGHTMARE and I'm still not 100% confident that
it works as it is supposed to. The generator, on the other hand, is
simple and follows the definition very closely. MUCH better.
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
I guess to be completely honest, there would be a docblock above the generator code as well. So 14 lines. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
I don't consider this a very good example. That's fine. You are entitled to your opinion. However, you didn't have to figure out the 106 lines of code that is the Post-Order iterator. That was a NIGHTMARE and I'm still not 100% confident that it works as it is supposed to. The generator, on the other hand, is simple and follows the definition very closely. MUCH better. Could you show the whole class that this public function belongs to? I don't see how this function fits in and compares at all to the class you linked. Brian. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
On Sun, Aug 12, 2012 at 8:25 PM, Brian Moon br...@moonspot.net wrote:
I don't consider this a very good example.
That's fine. You are entitled to your opinion. However, you didn't
have to figure out the 106 lines of code that is the Post-Order
iterator. That was a NIGHTMARE and I'm still not 100% confident that
it works as it is supposed to. The generator, on the other hand, is
simple and follows the definition very closely. MUCH better.
Could you show the whole class that this public function belongs to? I don't
see how this function fits in and compares at all to the class you linked.
Brian.
It renders the iterator class completely obsolete; it could be part of
the Binary Search Tree class.
If you still wanted a separate class (separation of concerns, perhaps)
it would look something like this (not sure on exact yield syntax):
class PostOrderIterator implements IteratorAggregate {
/**
* @var BinaryTree
*/
protected $root;
public function __construct(BinaryTree $root) {
$this-root = $root;
}
/**
* @link http://php.net/manual/en/iteratoraggregate.getiterator.php
* @return Traversable
*/
public function getIterator() {
yield $this-traverse($this-root);
}
private function traverse(BinaryTree $node) {
if ($node-left) {
yield* $node-left;
}
if ($node-right) {
yield* $node-right;
}
yield $node-value;
}
}
All this does is take the node that acts as root in it's constructor
and then calls the generator with root as the starting point. That's
34 lines of code including comments and whitespace.
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] [RFC] Generators
Brian, On Sun, Aug 12, 2012 at 4:08 PM, Brian Moon br...@moonspot.net wrote: Hi Nikita, I admit, I have ignored these threads as there was no RFC. So, some of this may have been covered. There was an RFC in those posts... It was just being iterated over. Do you have a good example usage other than a file? I don't find fopen/fgets/fclose all that complicated. What are the other valid use cases for such a thing? Here's a quick set of examples: http://blog.ircmaxell.com/2012/07/what-generators-can-do-for-you.html Also, not allowing rewinding is unintuitive for something that is an iterator in PHP. If I can foreach() it and I can call next() on it, I expect to be able to reset() it as well. IMO, you would need to issue a FATAL PHP error if that simply is not allowed. Or you have to have a second syntax for what to do in that case. At that point, you are implementing Iterator. I partially agree. rewinding the generator might be possible, but it's hard to tell in those cases. It's hard to tell if resetting it is even possible from the code level. So I'm pretty much OK with living with that restriction for the time being... While I am glad that PHP has borrowed syntax from many languages, I find the yield keyword to be very WTF when I first saw it. It is also poorly explained in your RFC. You use terms like sending and receiving. That does not say returns from function execution to me. I had to basically figure it out from the code examples. It's absolutely something that takes getting used to. But it's also quite intuitive once you think about it. You're not returning back to the parent scope (exiting your scope), you're yielding a value to the parent scope, expecting to continue on later (think of it as a stop light where you let the other code run for a while until you go to the next one). Lastly, because you cite needing this for sending data to PHPUnit, I think this is a user land problem and not a core problem. In about 5 minutes I implemented a reusable Generator object that does exactly what you need. http://pastebin.com/V336rEpR At least, it does what your examples show you need. Again, file IO is very easy and perhaps that example does not explain your true need very well. Well, there's one thing that should be made clear. There's nothing (and I mean that) that generators provide that you can't do already. You can build iterators that do exactly the same thing. The trick that generators provide you is a really short, and really easy way of implementing iterators. You don't need to worry about maintaining state or anything like that. All you need to do is code it how you would normally, and replace your executing code with a yield. No more needing to split out and maintain state in weird ways, or using literally hundreds of lines of code (because of the boilerplate involved) where a dozen would do... Anthony
