[PERFORM] Proposal for unlogged tables

2016-01-04 Thread Mark Zealey
I've recently been doing some performance testing with unlogged tables vs logged tables on 9.5-rc1. Basically we're trying to do big loads of data into the database periodically. If on the very rare occasion the server crashes half way through the import it's no big deal so I've been looking

Re: [PERFORM] Proposal for unlogged tables

2016-01-04 Thread Yves Dorfsman
On 2016-01-04 02:59, Mark Zealey wrote: > shutdown all unlogged tables will be truncated. Obviously with 9.5 we can now > alter tables to be logged/unlogged after insert but this will still write all > the inserts into the WAL. I haven't tried, but won't converting an unlogged table into a logged

Re: [PERFORM] Proposal for unlogged tables

2016-01-04 Thread Mark Zealey
On 04/01/16 16:27, Yves Dorfsman wrote: I haven't tried, but won't converting an unlogged table into a logged table write all the inserts at once instead of once per insert? Or are you wanting to do more bulk insert into that table later? Are you trying to avoid running a CHECKPOINT? Are you

Re: [PERFORM] Proposal for unlogged tables

2016-01-04 Thread Mark Zealey
On 04/01/16 18:12, Andres Freund wrote: Pages containing data of unlogged tables aren't ever flushed to disk unless a) a shutdown checkpoint is performed b) a buffer containing data from an unlogged table is used for something else c) the database being copied is the the source of a CREATE

Re: [PERFORM] Proposal for unlogged tables

2016-01-04 Thread Andres Freund
On 2016-01-04 19:12:22 +0200, Mark Zealey wrote: > If there was a command to flush a specific unlogged table to disk it would > work around all these issues no? Perhaps if you marked the table as read > only at the same time it would flush it to disk and ensure no more data > could be written to

Re: [PERFORM] Proposal for unlogged tables

2016-01-04 Thread Andres Freund
On 2016-01-04 16:38:40 +0200, Mark Zealey wrote: > I don't know how the internals work but unlogged tables definitely flushed > to disk and persist through normal server restarts. It is just according to > the docs if the server ever has an unclean shutdown the tables are truncated > even if they