Re: pg full text search very slow for Chinese characters

2019-09-10 Thread Kyotaro Horiguchi
Hi. At Tue, 10 Sep 2019 18:42:26 +0200 (CEST), Andreas Joseph Krogh wrote in > På tirsdag 10. september 2019 kl. 18:21:45, skrev Tom Lane > >: Jimmy Huang writes: > > I tried pg_trgm and my own customized token parser >

Re: pg full text search very slow for Chinese characters

2019-09-10 Thread Andreas Joseph Krogh
På tirsdag 10. september 2019 kl. 18:21:45, skrev Tom Lane mailto:t...@sss.pgh.pa.us>>: Jimmy Huang writes: > I tried pg_trgm and my own customized token parser https://github.com/huangjimmy/pg_cjk_parser pg_trgm is going to be fairly useless for indexing text that's mostly multibyte

Re: pg full text search very slow for Chinese characters

2019-09-10 Thread Tom Lane
Jimmy Huang writes: > I tried pg_trgm and my own customized token parser > https://github.com/huangjimmy/pg_cjk_parser pg_trgm is going to be fairly useless for indexing text that's mostly multibyte characters, since its unit of indexable data is just 3 bytes (not characters). I don't know of

Re: pg full text search very slow for Chinese characters

2019-09-10 Thread Jimmy Huang
Lewis 日期: 2019年9月11日 星期三 上午12:11 收件人: 黄 少君 抄送: "pgsql-general@lists.postgresql.org" 主题: Re: pg full text search very slow for Chinese characters >My postgres instance is based on docker image postgres:11 and runs on my >MacBook Pro i7 16GB. How much ram and such did you

Re: pg full text search very slow for Chinese characters

2019-09-10 Thread Cory Nemelka
On Tue, Sep 10, 2019 at 10:11 AM Michael Lewis wrote: > >My postgres instance is based on docker image postgres:11 and runs on my > MacBook Pro i7 16GB. > > How much ram and such did you give to this vm? > > > >To my surprise, postgres 11 is extremely slow when creating a full text > index. I

Re: pg full text search very slow for Chinese characters

2019-09-10 Thread Michael Lewis
>My postgres instance is based on docker image postgres:11 and runs on my MacBook Pro i7 16GB. How much ram and such did you give to this vm? >To my surprise, postgres 11 is extremely slow when creating a full text index. I added a column of tsvector type and tried to create an index on that

pg full text search very slow for Chinese characters

2019-09-10 Thread Jimmy Huang
Hi Team, Can anyone shed some light on why postgres 11 is extremely slow in my case? I am making a mirror of zh.wikisource.org and I have downloaded 303049 pages and stored them in a postgres 11 database. My postgres instance is based on docker image postgres:11 and runs on my MacBook Pro i7