Re: [HACKERS] Optimizing pglz compressor

2013-07-02 Thread Amit Kapila
On Monday, July 01, 2013 1:36 PM Heikki Linnakangas wrote: On 26.06.2013 16:37, Amit Kapila wrote: On Wednesday, June 26, 2013 2:15 AM Heikki Linnakangas wrote: Can you also try the attached patch, please? It's the same as before, but in this version, I didn't replace the prev and next

Re: [HACKERS] Optimizing pglz compressor

2013-07-01 Thread Heikki Linnakangas
On 26.06.2013 16:37, Amit Kapila wrote: On Wednesday, June 26, 2013 2:15 AM Heikki Linnakangas wrote: Can you also try the attached patch, please? It's the same as before, but in this version, I didn't replace the prev and next pointers in PGLZ_HistEntry struct with int16s. That avoids some

Re: [HACKERS] Optimizing pglz compressor

2013-07-01 Thread Bruce Momjian
On Mon, Jul 1, 2013 at 11:05:37AM +0300, Heikki Linnakangas wrote: On 26.06.2013 16:37, Amit Kapila wrote: On Wednesday, June 26, 2013 2:15 AM Heikki Linnakangas wrote: Can you also try the attached patch, please? It's the same as before, but in this version, I didn't replace the prev and

Re: [HACKERS] Optimizing pglz compressor

2013-06-26 Thread Amit Kapila
On Wednesday, June 26, 2013 2:15 AM Heikki Linnakangas wrote: On 19.06.2013 14:01, Amit Kapila wrote: Observations -- 1. For small data perforamce is always good with patch. 2. For random small/large data performace is good. 3. For medium and large text and same byte

Re: [HACKERS] Optimizing pglz compressor

2013-06-25 Thread Heikki Linnakangas
On 19.06.2013 14:01, Amit Kapila wrote: Observations -- 1. For small data perforamce is always good with patch. 2. For random small/large data performace is good. 3. For medium and large text and same byte data(3K,5K text, 10K,100K,500K same byte), performance is degraded. Wow,

Re: [HACKERS] Optimizing pglz compressor

2013-06-19 Thread Amit Kapila
On Tuesday, March 05, 2013 7:03 PM Heikki Linnakangas wrote: I spent some more time on this, and came up with the attached patch. It includes the changes I posted earlier, to use indexes instead of pointers in the hash table. In addition, it makes the hash table size variable, depending on

Re: [HACKERS] Optimizing pglz compressor

2013-03-18 Thread Daniel Farina
On Wed, Mar 6, 2013 at 6:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With these tweaks, I was able to make pglz-based delta encoding perform roughly as well as Amit's patch. Out of curiosity, do we know how

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Joachim Wieland
On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With these tweaks, I was able to make pglz-based delta encoding perform roughly as well as Amit's patch. Out of curiosity, do we know how pglz compares with other algorithms, e.g. lz4 ? -- Sent via

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Merlin Moncure
On Wed, Mar 6, 2013 at 8:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With these tweaks, I was able to make pglz-based delta encoding perform roughly as well as Amit's patch. Out of curiosity, do we know how

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Andres Freund
On 2013-03-06 09:36:19 -0600, Merlin Moncure wrote: On Wed, Mar 6, 2013 at 8:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With these tweaks, I was able to make pglz-based delta encoding perform roughly as

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Jeff Janes
On Wed, Mar 6, 2013 at 8:53 AM, Andres Freund and...@2ndquadrant.comwrote: On 2013-03-06 09:36:19 -0600, Merlin Moncure wrote: On Wed, Mar 6, 2013 at 8:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Andres Freund
On 2013-03-06 09:08:10 -0800, Jeff Janes wrote: On Wed, Mar 6, 2013 at 8:53 AM, Andres Freund and...@2ndquadrant.comwrote: On 2013-03-06 09:36:19 -0600, Merlin Moncure wrote: On Wed, Mar 6, 2013 at 8:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Merlin Moncure
On Wed, Mar 6, 2013 at 10:53 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-03-06 09:36:19 -0600, Merlin Moncure wrote: On Wed, Mar 6, 2013 at 8:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: With

Re: [HACKERS] Optimizing pglz compressor

2013-03-06 Thread Andres Freund
On 2013-03-06 11:31:06 -0600, Merlin Moncure wrote: On Wed, Mar 6, 2013 at 10:53 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-03-06 09:36:19 -0600, Merlin Moncure wrote: On Wed, Mar 6, 2013 at 8:32 AM, Joachim Wieland j...@mcknight.de wrote: On Tue, Mar 5, 2013 at 8:32 AM,

Re: [HACKERS] Optimizing pglz compressor

2013-03-05 Thread Heikki Linnakangas
I spent some more time on this, and came up with the attached patch. It includes the changes I posted earlier, to use indexes instead of pointers in the hash table. In addition, it makes the hash table size variable, depending on the length of the input. This further reduces the startup cost

[HACKERS] Optimizing pglz compressor

2013-03-01 Thread Heikki Linnakangas
I spotted some low-hanging fruit in the pglz compression routine. It uses a hash table to keep track of string prefixes it has seen: #define PGLZ_HISTORY_LISTS 8192/* must be power of 2 */ #define PGLZ_HISTORY_SIZE 4096 /* -- * Statically allocated work

Re: [HACKERS] Optimizing pglz compressor

2013-03-01 Thread Alvaro Herrera
Heikki Linnakangas wrote: I spotted this while looking at Amit's WAL update delta encoding patch. My earlier suggestion to just use the pglz compressor for the delta encoding didn't work too well because the pglz compressor was too expensive, especially for small values. This patch might help

Re: [HACKERS] Optimizing pglz compressor

2013-03-01 Thread Heikki Linnakangas
On 01.03.2013 17:37, Alvaro Herrera wrote: Heikki Linnakangas wrote: In summary, this seems like a pretty clear win for short values, and a wash for long values. Not surprising, as this greatly lowers the startup cost of pglz_compress(). We're past feature freeze, but how would people feel

Re: [HACKERS] Optimizing pglz compressor

2013-03-01 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Surely we're not past feature freeze. If we were, we'd have to reject all remaining patches from the commitfest, which is not what we want to do at this stage, is it? Actually, I think we're getting very close to exactly that point- we're not