Re: insert and query performance on big string table with pg_trgm

2017-12-06 Thread Matthew Hall
> On Dec 5, 2017, at 11:23 PM, Sergei Kornilov wrote: > You has very slow (or busy) disks, not postgresql issue. Reading 6760 * 8KB > in 70 seconds is very bad result. > > For better performance you need better disks, at least raid10 (not raid5). > Much more memory in shared_buffers can help w

Re: insert and query performance on big string table with pg_trgm

2017-12-05 Thread Sergei Kornilov
>Buffers: shared hit=544 read=6760 dirtied=4034 >I/O Timings: read=69709.611 You has very slow (or busy) disks, not postgresql issue. Reading 6760 * 8KB in 70 seconds is very bad result. For better performance you need better disks, at least raid10 (not raid5). Much more memory in shared

Re: insert and query performance on big string table with pg_trgm

2017-12-05 Thread Matthew Hall
On Nov 21, 2017, at 12:05 AM, Matthew Hall wrote: >> Do you really need the artificial primary key, when you already have another >> column that would be used as the primary key? If you need to use this it a >> foreign key in another type, then very well might. But maintaining two >> unique i

Re: insert and query performance on big string table with pg_trgm

2017-11-25 Thread Gábor SZŰCS
Don't know if it would make PostgreSQL happier but how about adding a hash value column and creating the unique index on that one? May block some false duplicates but the unique index would be way smaller, speeding up inserts. 2017. nov. 25. 7:35 ezt írta ("Jeff Janes" ): > > > On Nov 21, 2017 00

Re: insert and query performance on big string table with pg_trgm

2017-11-24 Thread Jeff Janes
On Nov 21, 2017 00:05, "Matthew Hall" wrote: > Are all indexes present at the time you insert? It will probably be much faster to insert without the gin index (at least) and build it after the load. There is some flexibility on the initial load, but the updates in the future will require the d

Re: insert and query performance on big string table with pg_trgm

2017-11-21 Thread Matthew Hall
Hi Jeff, Thanks so much for writing. You've got some great points. > On Nov 20, 2017, at 5:42 PM, Jeff Janes wrote: > While I have not done exhaustive testing, from the tests I have done I've > never found gist to be better than gin with trgm indexes. Thanks, this helps considerably, as the do

Re: insert and query performance on big string table with pg_trgm

2017-11-20 Thread Jeff Janes
On Mon, Nov 20, 2017 at 2:54 PM, Matthew Hall wrote: While I have not done exhaustive testing, from the tests I have done I've never found gist to be better than gin with trgm indexes. > > Here is the table: > >Unlogged table "public.huge_table" >Column

insert and query performance on big string table with pg_trgm

2017-11-20 Thread Matthew Hall
Hello PGSQL experts, I've used your great database pretty heavily for the last 4 years, and during that time it's helped me to solve an amazingly wide variety of data challenges. Last week, I finally ran into something weird enough I couldn't figure it out by myself. I'm using a self-compiled copy