I have managed to split the 365GB file into 2GB chunks with the help of
'split' unix utility in mingw shell like so
split -C 2GB ways.txt
Then I imported the files into a clean database with the help of the
following cmd command
for /f %f in ('dir /b') do psql -U postgres -w -d osm -t -c "set
client_encoding TO 'UTF8'; copy ways from 'D:\ways\%f';"
The operation took ~3.5 hour which is the same as v10!Prior to that I set 'parallel_leader_participation = on' and 'synchronous_commit = off' in the config file and restarted the server. Then I logged into the psql interactive terminal and ran ALTER TABLE ONLY ways ADD CONSTRAINT pk_ways PRIMARY KEY (id); It took 1h 10m which is 30m faster than with the default settings (after 'type' commad if it really matters) but still 3 times slower than in v10. Regards пн, 2 дек. 2019 г. в 12:04, Laurenz Albe <laurenz.a...@cybertec.at>: > On Sat, 2019-11-30 at 22:47 +0300, Eugene Podshivalov wrote: > > It sounds strange but the "type" is indeed impacting the overall > > performance somehow. > > I've just tried to execute the following sequence of commands on a > > fresh new database with PostreSQL v10 and both the copy and primary > > key commands performed as slow as in v11 and 12. > > > > SET synchronous_commit TO OFF; > > SET client_encoding TO 'UTF8'; > > COPY ways FROM program 'cmd /c "type D:\ways.txt"'; > > ALTER TABLE ONLY ways ADD CONSTRAINT pk_ways PRIMARY KEY (id); > > > > Regards > > > > пт, 29 нояб. 2019 г. в 15:22, Eugene Podshivalov <yauge...@gmail.com>: > > > Laurenz, > > > There is no way to run copy without the "type" on v11. See this thread > > > > https://www.postgresql.org/message-id/CAKHmqNCfTMM6%3DPqc6RUMEQ_2BPfo5KGGG-0fzRXZCVooo%3DwdNA%40mail.gmail.com > > > > > > My machine is running on NVMe disks, so the I/O subsystem very strong. > > > The 100% overload is not constant but periodical, as if there are some > > > kind of dumps for recovery performed in the background. > > Is it an option to split the file into parts of less than 2GB in size? > > Yours, > Laurenz Albe > -- > Cybertec | https://www.cybertec-postgresql.com > >
