Thank you for your answer!
On 5/4/22 13:33, Ben Kochie wrote:
That's not correct, the author seems to be unaware of the difference
between fsync and fdatasync[0].
The TSDB's WAL calls an internal sync(), which is implemented as
fdatasync() by default every minute on a timer.
https://github.com/prometheus/prometheus/blob/2381d7be57314df63c17291f22e5e770b44a70f3/tsdb/wal.go#L699-L702
It also calls the sync() when closing out individual WAL segments.
[0]: https://linux.die.net/man/2/fdatasync
On Wed, May 4, 2022 at 12:23 PM Volker Dormeyer <vol...@ixolution.de>
wrote:
Hello all,
I have serious question regarding the following article:
https://valyala.medium.com/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704
It is about the statement as follows from the author:
“Prometheus calls fsync
<https://github.com/prometheus/tsdb/blob/bc3b0bd429153ab54662a930df3817e4f29d169e/wal/wal.go#L390>
only after big chunk of data (aka |segment|) is written into WAL,
so all
the |segment| data may be lost / corrupted on power loss before
|fsync|.
The data may be corrupted if the OS flushes a few pages with the
written
data to disk, but doesn’t flush the remaining pages. Prometheus
|fscync|’s segments every 2 hours by default, so a lot of data may be
corrupted on hardware reset.”
Is it still true? Can we shorten the interval between fsync calls?
Does
this make sense?
Thanks!
Best Regards,
Volker
--
You received this message because you are subscribed to the Google
Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to prometheus-users+unsubscr...@googlegroups.com
<mailto:prometheus-users%2bunsubscr...@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/4a8ac708-57c7-ea33-8973-ac2ce0ab1ca7%40ixolution.de.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/1166bca5-7535-4d4a-2f7b-0779cb9b80c2%40ixolution.de.
