Re: Best use of "open" context manager
On 06Jul2024 11:49, Rob Cliffe wrote:
try:
f = open(FileName) as f:
FileLines = f.readlines()
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
print("I do a lot of processing here")
# Many lines of code here .
What about this:
try:
f = open(FileName) as f:
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
with f:
... process the lines here ...
Remember, the `open()` call returns a file object _which can be used as
a context manager_. It is separate from the `with` itself.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
On 6/07/24 22:49, Rob Cliffe via Python-list wrote:
Consider this scenario (which I ran into in real life):
I want to open a text file and do a lot of processing on the lines
of that file.
If the file does not exist I want to take appropriate action, e.g.
print an error message and abort the program.
I might write it like this:
try:
with open(FileName) as f:
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
but this violates the principle that a "try" suite should be kept small,
so that only targeted exceptions are trapped,
Yes!
not to mention that having "try" and "except" far apart decreases
readability.
Uh-oh!
- and there's a bit of a hang-over for old-timers who had to take care
of file-locking within the application - we try to minimise 'time'
between opening a file and closing it (etc)!
As it seems the file is opened to read. Less relevant in this case, but
habits and styles of coding matter...
Or I might write it like this:
try:
f = open(FileName) as f:
FileLines = f.readlines()
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
print("I do a lot of processing here")
# Many lines of code here .
but this loses the benefits of using "open" as a context manager,
and would also be unacceptable if the file was too large to read into
memory.
So, now there are two concerns:
1 FileNotFoundError, and
2 gradual processing to avoid memory-full
- added to remembering to close the file.
Really I would like to write something like
try:
with open(FileName) as f:
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
else: # or "finally:"
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
but this of course does not work because by the time we get to "for ln
in f:" the file has been closed so we get
ValueError: I/O operation on closed file
I could modify the last attempt to open the file twice, which would
work, but seems like a kludge (subject to race condition, inefficient).
Is there a better / more Pythonic solution?
Idea 1: invert the exception handling and the context-manager by writing
a custom context-manager class which handles FileNotFoundError
internally. Thus, calling-code becomes:
with...
for...
processing
Idea 2: incorporate idea of encapsulating "processing" into a
(well-named) function to shorten the number of lines-of-code inside the
with-suite.
Idea 3: consider using a generator to 'produce' lines of data
one-at-a-time. Remember that whilst context-managers and generators are
distinct concepts within Python, they are quite similar in many ways.
So, a custom generator could work like a context-manager or 'wrap' a
context-manager per Idea 1.
Building a custom-class (Idea 1 or Idea 3) enables the components to be
kept together, per the ideal. It keeps the try-except components close
and easy to relate. It is Pythonic (in the OOP style).
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
My thoughts is that if the "many lines of code" puts the except to far
from the try, then perhaps it would have made sense to factor out some
part there into a function.
Perhaps like:
try:
with open(FileName) as f:
for ln in f{
process(ln)
except FileNotFoundError:
print(f"File {FileName} not found:")
sys.exit()
Now the "process" function has been factored out and can be well
documented as to what it is doing on each line, and this code can be
documented as running process on each line of the file.
On 7/6/24 6:49 AM, Rob Cliffe via Python-list wrote:
Consider this scenario (which I ran into in real life):
I want to open a text file and do a lot of processing on the lines
of that file.
If the file does not exist I want to take appropriate action, e.g.
print an error message and abort the program.
I might write it like this:
try:
with open(FileName) as f:
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
but this violates the principle that a "try" suite should be kept
small, so that only targeted exceptions are trapped,
not to mention that having "try" and "except" far apart decreases
readability.
Or I might write it like this:
try:
f = open(FileName) as f:
FileLines = f.readlines()
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
print("I do a lot of processing here")
# Many lines of code here .
but this loses the benefits of using "open" as a context manager,
and would also be unacceptable if the file was too large to read into
memory.
Really I would like to write something like
try:
with open(FileName) as f:
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
else: # or "finally:"
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
but this of course does not work because by the time we get to "for ln
in f:" the file has been closed so we get
ValueError: I/O operation on closed file
I could modify the last attempt to open the file twice, which would
work, but seems like a kludge (subject to race condition, inefficient).
Is there a better / more Pythonic solution?
Best wishes
Rob Cliffe
--
Richard Damon
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
On 7/6/2024 6:49 AM, Rob Cliffe via Python-list wrote:
Consider this scenario (which I ran into in real life):
I want to open a text file and do a lot of processing on the lines
of that file.
If the file does not exist I want to take appropriate action, e.g.
print an error message and abort the program.
I might write it like this:
try:
with open(FileName) as f:
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
but this violates the principle that a "try" suite should be kept small,
so that only targeted exceptions are trapped,
not to mention that having "try" and "except" far apart decreases
readability.
Or I might write it like this:
try:
f = open(FileName) as f:
FileLines = f.readlines()
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
print("I do a lot of processing here")
# Many lines of code here .
but this loses the benefits of using "open" as a context manager,
and would also be unacceptable if the file was too large to read into
memory.
Really I would like to write something like
try:
with open(FileName) as f:
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
else: # or "finally:"
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
but this of course does not work because by the time we get to "for ln
in f:" the file has been closed so we get
ValueError: I/O operation on closed file
I could modify the last attempt to open the file twice, which would
work, but seems like a kludge (subject to race condition, inefficient).
Is there a better / more Pythonic solution?
I usually read the file into a sequence of lines and then leave the
open() as soon as possible. Something like this:
FILENAME = 'this_is_an_example.txt'
lines = None
if os.path.exists(FILENAME):
with open(FILENAME) as f:
lines = f.readlines()
# do something with lines
Of course, if you want to read a huge number of lines you will need to
be more thoughtful about it. Or make all the processing within the
open() block be a function. Then you just have one more line in the block.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
On Sat, 6 Jul 2024 at 11:55, Rob Cliffe via Python-list
wrote:
>
> Consider this scenario (which I ran into in real life):
> I want to open a text file and do a lot of processing on the lines
> of that file.
> If the file does not exist I want to take appropriate action, e.g.
> print an error message and abort the program.
> I might write it like this:
>
> try:
> with open(FileName) as f:
> for ln in f:
> print("I do a lot of processing here")
> # Many lines of code here .
> except FileNotFoundError:
> print(f"File {FileName} not found")
> sys.exit()
>
> but this violates the principle that a "try" suite should be kept small,
> so that only targeted exceptions are trapped,
> not to mention that having "try" and "except" far apart decreases
> readability.
This is catching a targeted exception (FileNotFoundError) so I think
it is fine. If the intention is just to call sys.exit() on error then
I wouldn't worry too much about having too much code in the try. Just
make sure that you do this in any other place where you open a file as
well.
One possible improvement is that you could catch the exception and use
its filename attribute:
except FileNotFoundError as e
print(f"File {e.filename} not found")
That way if you did catch the wrong FileNotFoundError then at least
you print the correct filename.
Alternatively:
except FileNotFoundError as e
if e.filename != FileName:
raise # re-raise if not the intended exception
print(f"File {e.filename} not found")
For readability I would just move the many lines of code into a
separate function.
The reason to avoid having too much code in the try mainly applies to
situations where you are going to do something other than call
sys.exit() and the exception is overly generic like ValueError or
TypeError. If the exception can easily be raised by a bug or something
other than the intended cause then it is bad to catch exceptions
around a larger block of code.
If it is expected that the caller of a function might have good reason
to catch the exception and handle it somehow then it is better to make
a dedicated exception class and raise that instead. When there is only
one place in the code that raises a particular exception type and only
one place that catches it then it is usually going to be clear that
you are catching the expected exception.
--
Oscar
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
On 06/07/2024 11:49, Rob Cliffe via Python-list wrote:
> If the file does not exist I want to take appropriate action, e.g.
> print an error message and abort the program.
> I might write it like this:
>
> try:
> with open(FileName) as f:
> for ln in f:
> print("I do a lot of processing here")
> # Many lines of code here .
> except FileNotFoundError:
> print(f"File {FileName} not found")
> sys.exit()
>
> but this violates the principle that a "try" suite should be kept small,
The try is small, it only has a single statement inside.
The compound block inside that statement should have its
own try/ecxepts but the outer one really only applies to
the with statement.
I certainly prefer this option to any of the others presented.
> not to mention that having "try" and "except" far apart decreases
> readability.
This is a valid concern although that's part of the reason
we use indentation. Compared to early BASIC and FORTRAN with massive
GOSUB type leaps (and often no indentation) it's very readable!
But its still preferable to having the multi-level indents below
or having to remember to close the file when finished (and
ensure that all possible paths do so.
> try:
> f = open(FileName) as f:
> FileLines = f.readlines()
> except FileNotFoundError:
> print(f"File {FileName} not found")
> sys.exit()
> # I forgot to put "f.close()" here -:)
Exactly! That's why using with is safer even if the
except is detached from the try.
You are also reading the entire file into memory which could
be an issue. And you are not catching any errors in the
read operations because the except only covers a missing
file.
> Really I would like to write something like
>
> try:
> with open(FileName) as f:
> except FileNotFoundError:
> print(f"File {FileName} not found")
> sys.exit()
> else: # or "finally:"
> for ln in f:
> print("I do a lot of processing here")
> # Many lines of code here .
I find that much less readable because the file handling
block is a long way from the open file line and has extra
control statements to mentally negotiate.
The advantage of the original version is that you can
ignore errors and read the code easily. You only need
to find the except clause if you need to know how errors
will be dealt with. But if comprehending the core
functionality of the code you can just ignore all
the error handling blocks.
> Is there a better / more Pythonic solution?
All IMHO of course, but I think the current implementation
is the best of the options presented.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best use of "open" context manager
On 2024-07-06 at 11:49:06 +0100, Rob Cliffe via Python-list wrote: > Is there a better / more Pythonic solution? https://docs.python.org/3/library/fileinput.html At least this attempts to abstract the problem of iterating over a file (or multiple files) into a library routine. I've used it a little, but I don't know the full depths of your use case and/or requirements. HTH, Dan -- https://mail.python.org/mailman/listinfo/python-list
Best use of "open" context manager
Consider this scenario (which I ran into in real life):
I want to open a text file and do a lot of processing on the lines
of that file.
If the file does not exist I want to take appropriate action, e.g.
print an error message and abort the program.
I might write it like this:
try:
with open(FileName) as f:
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
but this violates the principle that a "try" suite should be kept small,
so that only targeted exceptions are trapped,
not to mention that having "try" and "except" far apart decreases
readability.
Or I might write it like this:
try:
f = open(FileName) as f:
FileLines = f.readlines()
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
print("I do a lot of processing here")
# Many lines of code here .
but this loses the benefits of using "open" as a context manager,
and would also be unacceptable if the file was too large to read into
memory.
Really I would like to write something like
try:
with open(FileName) as f:
except FileNotFoundError:
print(f"File {FileName} not found")
sys.exit()
else: # or "finally:"
for ln in f:
print("I do a lot of processing here")
# Many lines of code here .
but this of course does not work because by the time we get to "for ln
in f:" the file has been closed so we get
ValueError: I/O operation on closed file
I could modify the last attempt to open the file twice, which would
work, but seems like a kludge (subject to race condition, inefficient).
Is there a better / more Pythonic solution?
Best wishes
Rob Cliffe
--
https://mail.python.org/mailman/listinfo/python-list
