From: "Fujii Masao" <masao.fu...@gmail.com>
What's the main purpose of this tool? If it's for WAL archiving, the tool name
"pg_copy" sounds too generic. We already have pg_archivecleanup, so maybe
"pg_archivecopy" or something is better for the consistency?

pg_copy in the patch copies the file to the destination in a
straightforward way,
i.e., directly copies the file to the dest file with actual name. This can cause the problem which some people reported. The problem is that, when the server crashes while WAL file is being archived by cp command, its partially-filled WAL file remains at the archival area. This half-baked archive file can cause various troubles. To address this, WAL file needs to be copied to the temporary file at first, then renamed to the actual name. I think that pg_copy should
copy the WAL file in that way.

I intended to make pg_copy a straightforward replacement of cp/copy, which complements the missing sync. Direct I/O and posix_fadvice() feature may be convenient but not essential for this utility. cp/copy doesn't copy to a temporary file, and the problem can be solved easily by mv/move. I wanted to keep pg_copy as generic as cp/copy, so that it can be used by some advanced features in the future, e.g. comprehensive backup/recovery management like RMAN (this example may not be best) when it's integrated into the core.

With that said, copying to a temporary file like <dest>.tmp and renaming it to <dest> sounds worthwhile even as a basic copy utility. I want to avoid copying to a temporary file with a fixed name like _copy.tmp, because some advanced utility may want to run multiple instances of pg_copy to copy several files into the same directory simultaneously. However, I'm afraid multiple <dest>.tmp files might continue to occupy disk space after canceling copy or power failure in some use cases, where the copy of the same file won't be retried. That's also the reason why I chose to not use a temporary file like cp/copy.

Currently pg_copy always syncs the archive file, and there is no way to disable that. But I'm sure that not everyone want to sync the archive file. So I think
that it's better to add the option specifying whether to sync the file
or not, into
pg_copy.

pg_copy is for copying a file reliably by syncing. If sync is not necessary, people can use cp/copy.


Some users might want to specify whether to call posix_fadvise or not because
they might need to re-read the archvied files just after the archiving.
For example, network copy of the archived files from the archive area to
remote site for disaster recovery.

This sounds reasonable. Do you have an idea on the switch name and the default behavior?


Do you recommend to use pg_copy for restore_command? If yes, it also should be documented. And in the WAL restore case, the restored WAL files are re-read
soon by recovery, so posix_fadvise is not good in that case.

Direct I/O and posix_fadvise are used only for destination file. But why not
source file? That might be useful especially for restore_command case.

No, I don't think it's necessary to use pg_copy for restore_command.


At last, the big question is, is there really no OS command which provides
the same functionality as pg_copy does? If there is, I'd like to avoid duplicate
work basically.

If there exists such a command available in the standard OS installation, I want to use it.

Regards
MauMau



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Reply via email to