Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Andrey Borodin
> 21 апр. 2021 г., в 21:01, Stefan Keller написал(а): > > What's needed - and what many here as well as the "ML-In-Databases" > paper from Kraska et al. (2021) are saying - is, that a new index > (like a learned index) should be implemented as a PostgreSQL > extension. BTW, you don't have to

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Stefan Keller
Mi., 21. Apr. 2021, 11:16 Uhr, Oleg Bartunov wrote: > Have you seen recent paper "Benchmarking Learned Indexes" ? Yes. I skipped it after that this benchmark "just" compares the algorithm implementations. What's needed - and what many here as well as the "ML-In-Databases" paper from Kraska et al

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Bruce Momjian
On Wed, Apr 21, 2021 at 10:52:19AM +0200, Stefan Keller wrote: > Di., 20. Apr. 2021 23:50 Tom Lane wrote: > > There's enough support these days that you can build a new index > > type as an extension, without touching the core code at all. > > Thanks. I'm ramping up knowledge about extending PG w

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Oleg Bartunov
On Tue, Apr 20, 2021 at 8:56 PM Stefan Keller wrote: > Dear Olegs, dear Nikolay, dear all > > Allow me to revive this thread: > > Are there any advances in a learned index for PostgreSQL? > > Background: I'm trying to benchmark those experimental indices. For > this I did some bibliography work (

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Stefan Keller
Di., 20. Apr. 2021 23:50 Tom Lane wrote: > There's enough support these days that you can build a new index > type as an extension, without touching the core code at all. Thanks. I'm ramping up knowledge about extending PG with C++. I'm still interested to understand in principle what an index h

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 2:29 PM Stefan Keller wrote: > Just for the records: A learned index as no more foreknowledge about > the dataset as other indices. Maybe. ML models are famously prone to over-interpreting training data. In any case I am simply not competent to assess how true this is. >

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Tom Lane
Stefan Keller writes: > I'd give learned indexes at least a change to provide a > proof-of-concept. And I want to learn more about the requirements to > be accepted as a new index (before undergoing month's of code > sprints). There's enough support these days that you can build a new index type

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Stefan Keller
Just for the records: A learned index as no more foreknowledge about the dataset as other indices. I'd give learned indexes at least a change to provide a proof-of-concept. And I want to learn more about the requirements to be accepted as a new index (before undergoing month's of code sprints). A

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 12:51 PM Jonah H. Harris wrote: >> Maybe I'll be wrong about learned indexes - who knows? But the burden >> of proof is not mine. I prefer to spend my time on things that I am >> reasonably confident will work out well ahead of time. > > > Agreed on all of your takes, Peter

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Jonah H. Harris
On Tue, Apr 20, 2021 at 3:45 PM Peter Geoghegan wrote: > On Tue, Apr 20, 2021 at 12:35 PM Chapman Flack > wrote: > > How would showing that to be true for data structure X be different from > > making a case for data structure X? > > You don't have to understand the theoretical basis of B-Tree i

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 12:35 PM Chapman Flack wrote: > How would showing that to be true for data structure X be different from > making a case for data structure X? You don't have to understand the theoretical basis of B-Tree indexes to see that they work well. In fact, it took at least a decad

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Chapman Flack
On 04/20/21 15:24, Peter Geoghegan wrote: > data structures that work well don't need anybody to make a case for them. > They simply work well for the task they were designed for. How would showing that to be true for data structure X be different from making a case for data structure X? Regards,

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 11:18 AM Andrey Borodin wrote: > BTW take a look into PGM [0]. I'm slowly working on implementing it. > I think it is kind of straightforward to implement it as extension. > I've started from forking B-tree[1]. I've removed support of anything that is > not int4. > Then I

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Andrey Borodin
> 20 апр. 2021 г., в 22:56, Stefan Keller написал(а): > > Are there any advances in a learned index for PostgreSQL? BTW take a look into PGM [0]. I'm slowly working on implementing it. I think it is kind of straightforward to implement it as extension. I've started from forking B-tree[1]. I'v

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Stefan Keller
Dear Olegs, dear Nikolay, dear all Allow me to revive this thread: Are there any advances in a learned index for PostgreSQL? Background: I'm trying to benchmark those experimental indices. For this I did some bibliography work (see below). Fun fact: Not only Postgres people love high-proof drink

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2017-12-12 Thread Oleg Ivanov
On 12/12/2017 04:33 PM, Oleg Bartunov wrote: On Mon, Dec 11, 2017 at 11:11 PM, Nikolay Samokhvalov wrote: Very interesting read: https://arxiv.org/abs/1712.01208 HN discussion: https://news.ycombinator.com/item?id=15894896 Some of the comments (from Twitter https://twitter.com/schrockn/status

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2017-12-12 Thread Oleg Bartunov
On Mon, Dec 11, 2017 at 11:11 PM, Nikolay Samokhvalov wrote: > Very interesting read: https://arxiv.org/abs/1712.01208 > > HN discussion: https://news.ycombinator.com/item?id=15894896 > > Some of the comments (from Twitter > https://twitter.com/schrockn/status/940037656494317568): "Jeff Dean and c

ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2017-12-11 Thread Nikolay Samokhvalov
Very interesting read: https://arxiv.org/abs/1712.01208 HN discussion: https://news.ycombinator.com/item?id=15894896 Some of the comments (from Twitter https://twitter.com/schrockn/status/940037656494317568): "Jeff Dean and co at GOOG just released a paper showing how machine-learned indexes can