On 05Mar2023 10:38, aapost <aap...@idontexist.club> wrote:
Additionally (not sure if this still applies):
flush() does not necessarily write the file’s data to disk. Use flush()
followed by os.fsync() to ensure this behavior.
Yes. You almost _never_ need or want this behaviour. A database tends to
fsync at the end of a transaction and at other critical points.
However, once you've `flush()`ed the file the data are then in the hands
of the OS, to get to disc in a timely but efficient fashion. Calling
fsync(), like calling flush(), affects writing _efficiency_ by depriving
the OS (or for flush(), the Python I/O buffering system) the opportunity
to bundle further data efficiency. It will degrade the overall
performance.
Also, fsync() need not expedite the data getting to disc. It is equally
valid that it just blocks your programme _until_ the data have gone to
disc. I practice it probably does expedite things slightly, but the real
world effect is that your pogramme will gratuitously block anyway, when
it could just get on with its work, secure in the knowledge that the OS
has its back.
flush() is for causality - ensuring the data are on their way so that
some external party _will_ see them rather than waiting forever for data
with are lurking in the buffer. If that external party, for you, is an
end user tailing a log file, then you might want to flush(0 at the end
of every line. Note that there is a presupplied line-buffering mode you
can choose which will cause a file to flush like that for you
automatically.
So when you flush is a policy decision which you can make either during
the programme flow or to a less flexible degree when you open the file.
As an example of choosing-to-flush, here's a little bit of code in a
module I use for writing packet data to a stream (eg a TCP connection):
https://github.com/cameron-simpson/css/blob/00ab1a8a64453dc8a39578b901cfa8d1c75c3de2/lib/python/cs/packetstream.py#L624
Starting at line 640: `if Q.empty():` it optionally pauses briefly to
see if more packets are coming on the source queue. If another arrives,
the flush() is _skipped_, and the decision to flush made again after the
next packet is transcribed. In this way a busy source of packets can
write maximally efficient data (full buffers) as long as there's new
data coming from the queue, but if the queue is empty and stays empty
for more that `grace` seconds we flush anyway so that the receiver
_will_ still see the latest packet.
Cheers,
Cameron Simpson <c...@cskk.id.au>
--
https://mail.python.org/mailman/listinfo/python-list
