avi.e.gr...@gmail.com wrote:
That is an excellent point, Mark. Some of the proposed variants to the
requested problem, including mine, do indeed find all instances only to return
the first. This can use additional time and space but when done, some of the
overhead is also gone. What I mean is that a generator you create and invoke
once, generally sits around indefinitely in your session unless it leaves your
current range or something. It does only a part of the work and must remain
suspended and ready to be called again to do more.
It goes out of scope at the end of the function. Unless you return it
or store a reference to it elsewhere, it will then be deleted.
Or in this case, since the `first_word_beginning_with_e` function
doesn't even have a local reference to the generator (it is just created
and immediately passed as an argument to `next`), it goes out of scope
once the `next` function returns.
If you create a generator inside a function and the function returns,
presumably it can be garbage-collected.
Exactly. It probably doesn't even need to wait for garbage collection -
once the reference count is zero, it can be destroyed.
But if it is in the main body, I have to wonder what happen.
If you mean in the top-level module scope outside of any
function/method, then it would remain in memory until the process exits.
There seem to be several related scenarios to consider.
- You may want to find, in our example, a first instance. Right afterwards, you
want the generator to disassemble anything in use.
- You may want the generator to stick around and later be able to return the
next instance. The generator can only really go away when another call has been
made after the last available instance and it cannot look for more beyond some
end.
- Finally, you can call a generator with the goal of getting all instances such
as by asking it to populate a list. In such a case, you may not necessarily
want or need to use a generator expression and can use something
straightforward and possible cheaper.
Yes, so you create and assign it at an appropriate scope. In the
example here, it's just passed to `next` and then destroyed. Passing a
generator to the `list` constructor (or the `tuple` constructor in my
"FWIW") would behave similarly - you'd get the final list/tuple back,
but the generator would be destroyed once that call is done. If you
assigned it to a function-local variable, it would exist until the end
of that function.
What confuses the issue, for me, is that you can make fairly complex
calculations in python using various forms of generators that implement a sort
of just-in-time approach as generators call other generators which call yet
others and so on.
Yes, you can. It can be quite useful when used appropriately.
Imagine having folders full of files that each contain a data structure such as
a dictionary or set and writing functionality that searches for the first match
for a key in any of the dictionaries (or sets or whatever) along the way? Now
imagine that dictionary items can be a key value pair that can include the
value being a deeper dictionary, perhaps down multiple levels.
You could get one generator that generates folder names or opens them and
another that generates file names and reads in the data structure such as a
dictionary and yet another that searches each dictionary and also any
internally embedded dictionaries by calling another instance of the same
generator as much as needed.
You probably could do that. Personally, I probably wouldn't use
generators for that, or at least not custom ones - if you're talking
about iterating over directories and files on disk, I'd probably just
use `os.walk` (which probably is a generator) and iterate over that,
opening each file and doing whatever you want with the contents.
You can see how this creates and often consumes generators along the way as
needed and in a sense does the minimum amount of work needed to find a first
instance. But what might it leave open and taking up resources if not finished
in a way that dismantles it?
You'd need to make sure any files are closed (`with open(...)` helps
with that). If you're opening files within a generator, I'm pretty sure
you can do something like:
```
def iter_files(directory):
for filename in directory:
with open(filename) as f:
yield f
```
Then the file will be closed when the iterator leaves the `with` block
and moved on to the next item (presumably there's some mechanism for the
context manager's `__exit__` to be called if the generator is destroyed
without having iterated over the items - the whole point of using `with`
is that `__exit__` is guaranteed to be called whatever happens).
Other than that, the generators themselves would be destroyed once they
go out of scope. If there are no references to a generator left,
nothing is going to be able to call `next` (nor anything else) on it, so
no need for it to be kept hanging around in memory.
Perhaps worse, imagine doing the search in parallel and as sone as it is found
anywhere, ...
-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail....@python.org> On
Behalf Of Mark Bourne via Python-list
Sent: Thursday, April 4, 2024 3:04 PM
To: python-list@python.org
Subject: Re: A technique from a chatbot
Thomas Passin wrote:
On 4/2/2024 1:47 PM, Piergiorgio Sartor via Python-list wrote:
On 02/04/2024 19.18, Stefan Ram wrote:
Some people can't believe it when I say that chatbots improve
my programming productivity. So, here's a technique I learned
from a chatbot!
It is a structured "break". "Break" still is a kind of jump,
you know?
So, what's a function to return the first word beginning with
an "e" in a given list, like for example
[ 'delta', 'epsilon', 'zeta', 'eta', 'theta' ]
? Well it's
def first_word_beginning_with_e( list_ ):
for word in list_:
if word[ 0 ]== 'e': return word
. "return" still can be considered a kind of "goto" statement.
It can lead to errors:
def first_word_beginning_with_e( list_ ):
for word in list_:
if word[ 0 ]== 'e': return word
something_to_be_done_at_the_end_of_this_function()
The call sometimes will not be executed here!
So, "return" is similar to "break" in that regard.
But in Python we can write:
def first_word_beginning_with_e( list_ ):
return next( ( word for word in list_ if word[ 0 ]== 'e' ), None )
Doesn't look a smart advice.
. No jumps anymore, yet the loop is aborted on the first hit
It's worse than "not a smart advice". This code constructs an
unnecessary tuple, then picks out its first element and returns that.
I don't think there's a tuple being created. If you mean:
( word for word in list_ if word[ 0 ]== 'e' )
...that's not creating a tuple. It's a generator expression, which
generates the next value each time it's called for. If you only ever
ask for the first item, it only generates that one.
When I first came across them, I did find it a bit odd that generator
expressions look like the tuple equivalent of list/dictionary
comprehensions.
FWIW, if you actually wanted a tuple from that expression, you'd need to
pass the generator to tuple's constructor:
tuple(word for word in list_ if word[0] == 'e')
(You don't need to include an extra set of brackets when passing a
generator a the only argument to a function).
--
https://mail.python.org/mailman/listinfo/python-list
