Compressing temporary files

2021-09-11 Thread Andrey Borodin
Hi hackers! There's a lot of compression discussions nowadays. And that's cool! Recently Naresh Chainani in private discussion shared with me the idea to compress temporary files on disk. And I was thrilled to find no evidence of implementation of this interesting idea. I've prototyped Random A

Re: Compressing temporary files

2021-10-06 Thread Bruce Momjian
On Sat, Sep 11, 2021 at 05:31:37PM +0500, Andrey Borodin wrote: > How do you think: does it worth to pursue the idea? OLTP systems rarely rely > on data spilled to disk. > Are there any known good random access compressed file libs? So we could > avoid reinventing the wheel. > Maybe someone tried

Re: Compressing temporary files

2021-10-06 Thread Robert Haas
On Sat, Sep 11, 2021 at 8:31 AM Andrey Borodin wrote: > I've prototyped Random Access Compressed File for fun[0]. The code is very > dirty proof-of-concept. > I compress Buffile by one block at a time. There are directory pages to store > information about the size of each compressed block. If a

Re: Compressing temporary files

2021-10-06 Thread Tomas Vondra
Hi, On 9/11/21 2:31 PM, Andrey Borodin wrote: > Hi hackers! > > There's a lot of compression discussions nowadays. And that's cool! > Recently Naresh Chainani in private discussion shared with me the > idea to compress temporary files on disk. And I was thrilled to find > no evidence of implemen

Re: Compressing temporary files

2021-10-08 Thread Bharath Rupireddy
On Sat, Sep 11, 2021, 6:01 PM Andrey Borodin wrote: > > Hi hackers! > > There's a lot of compression discussions nowadays. And that's cool! > Recently Naresh Chainani in private discussion shared with me the idea to > compress temporary files on disk. > And I was thrilled to find no evidence of