Re: [HACKERS] Regarding GIN Fast Update Technique
On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote amitlangot...@gmail.com wrote: At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? I don't think there's any special handling for that case, nor do I think any is needed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Regarding GIN Fast Update Technique
On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote amitlangot...@gmail.com wrote: At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? I don't think there's any special handling for that case, nor do I think any is needed. Okay, aside from that case, what else would move those to the main structure? They (the entries in the unsorted pending list) are in the local memory (work_mem?) of the backend, right? -- Amit Langote -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Regarding GIN Fast Update Technique
On 2013-06-07 23:28:56 +0900, Amit Langote wrote: On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote amitlangot...@gmail.com wrote: At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? I don't think there's any special handling for that case, nor do I think any is needed. Okay, aside from that case, what else would move those to the main structure? They (the entries in the unsorted pending list) are in the local memory (work_mem?) of the backend, right? No, it's in the normal heap, pointed to by the metapage. Storing it in local memory would mean we would have to flush it out before commit. That part of gin is actually quite readable code, so I suggest looking there. Start with ginfast.c:ginHeapTupleFastInsert(). Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Regarding GIN Fast Update Technique
Amit Langote amitlangot...@gmail.com writes: Okay, aside from that case, what else would move those to the main structure? They (the entries in the unsorted pending list) are in the local memory (work_mem?) of the backend, right? No. If they were, it wouldn't be crash-safe. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Regarding GIN Fast Update Technique
On Fri, Jun 7, 2013 at 11:36 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-06-07 23:28:56 +0900, Amit Langote wrote: On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote amitlangot...@gmail.com wrote: At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? I don't think there's any special handling for that case, nor do I think any is needed. Okay, aside from that case, what else would move those to the main structure? They (the entries in the unsorted pending list) are in the local memory (work_mem?) of the backend, right? No, it's in the normal heap, pointed to by the metapage. Storing it in local memory would mean we would have to flush it out before commit. That part of gin is actually quite readable code, so I suggest looking there. Start with ginfast.c:ginHeapTupleFastInsert(). Thanks. -- Amit Langote -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Regarding GIN Fast Update Technique
On 07/06/13 16:39, Tom Lane wrote: Amit Langote amitlangot...@gmail.com writes: Okay, aside from that case, what else would move those to the main structure? They (the entries in the unsorted pending list) are in the local memory (work_mem?) of the backend, right? No. If they were, it wouldn't be crash-safe. Thats how it is, but if we could push in wishes, then I would wish that is woulndn't be crash-safe, and be flushed by the backends commit. The way it currently operates is that a random backend pays the penalty of other backends pushes to the pending-list and all queries pays the penalty of searching the pendinglist in queries. If the pending list were backend only it would not have to be searched by all queries since commit needs to flush it and random backends wouldn't be penalized. Allthough we'd still have the benefit of batching up gin-inserts over mulitiple changes to the index done within the same transaction. Jesper -- Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Regarding GIN Fast Update Technique
Hello, At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? -- Amit Langote -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
