Yakov,
This will work. However, I expect performance degradation with this
change. Disk storage has a limited number of I/O operations per second
on hardware level. List of already existing disk I/O activities (writing
to WAL work dir, copying from WAL work dir to WAL archive dir, writing
partition files during checkpoint) will be updated with a new one -
copying from WAL work dir to temp dir.
Best Regards,
Ivan Rakov
On 13.02.2018 21:35, Yakov Zhdanov wrote:
Ivan,
I do not want to create new files. As far as I know, now we copy segments
to archive dir before they get checkpointed. What I suggest is to copy them
to a temp dir under wal directory and then move to archive. In my
understanding at the time we copy the files to a temp folder all changes to
them are already fsynced.
Correct?
Yakov Zhdanov,
www.gridgain.com
2018-02-13 21:29 GMT+03:00 Ivan Rakov <ivan.glu...@gmail.com>:
Yakov,
I see the only one problem with your suggestion - number of
"uncheckpointed" segments is potentially unlimited.
Right now we have limited number (10) of file segments with immutable
names in WAL "work" directory. We have to keep this approach due to known
bug in XFS - fsync time is nearly twice bigger for recently created files.
Best Regards,
Ivan Rakov
On 13.02.2018 21:22, Yakov Zhdanov wrote:
I meant we still will be copying segment once and then will be moving it
to
archive which should not affect file system much.
Thoughts?
--Yakov
2018-02-13 21:19 GMT+03:00 Yakov Zhdanov <yzhda...@apache.org>:
Alex,
I remember we had some confusing behavior for WAL archive when archived
segments were required for successful recovery.
Is issue still present?
If yes, what if we copy "uncheckpointed" segments to a directory under
wal
directory and then move the segments to archive after checkpoint? Will
this
work?
Thanks!
--Yakov
