Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

> 21 апр. 2021 г., в 21:01, Stefan Keller написал(а): > > What's needed - and what many here as well as the "ML-In-Databases" > paper from Kraska et al. (2021) are saying - is, that a new index > (like a learned index) should be implemented as a PostgreSQL > ext

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

abases" paper from Kraska et al. (2021) are saying - is, that a new index (like a learned index) should be implemented as a PostgreSQL extension. Mi., 21. Apr. 2021, 15:46 Uhr, Bruce Momjian wrote: > The issue is that some index structures, like bitmap indexes, have very > poor concur

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Wed, Apr 21, 2021 at 10:52:19AM +0200, Stefan Keller wrote: > Di., 20. Apr. 2021 23:50 Tom Lane wrote: > > There's enough support these days that you can build a new index > > type as an extension, without touching the core code at all. > > Thanks. I'm ramping up knowledge about extending PG

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Tue, Apr 20, 2021 at 8:56 PM Stefan Keller wrote: > Dear Olegs, dear Nikolay, dear all > > Allow me to revive this thread: > > Are there any advances in a learned index for PostgreSQL? > > Background: I'm trying to benchmark those experimental indices. For > this I di

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

Peter Geoghegan : > > On Tue, Apr 20, 2021 at 2:29 PM Stefan Keller wrote: > > Just for the records: A learned index as no more foreknowledge about > > the dataset as other indices. > > Maybe. ML models are famously prone to over-interpreting training > data. In any case I

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Tue, Apr 20, 2021 at 2:29 PM Stefan Keller wrote: > Just for the records: A learned index as no more foreknowledge about > the dataset as other indices. Maybe. ML models are famously prone to over-interpreting training data. In any case I am simply not competent to assess ho

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

Stefan Keller writes: > I'd give learned indexes at least a change to provide a > proof-of-concept. And I want to learn more about the requirements to > be accepted as a new index (before undergoing month's of code > sprints). There's enough support these days that you can build a new index type

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

Just for the records: A learned index as no more foreknowledge about the dataset as other indices. I'd give learned indexes at least a change to provide a proof-of-concept. And I want to learn more about the requirements to be accepted as a new index (before undergoing month's of code sprints

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Tue, Apr 20, 2021 at 12:51 PM Jonah H. Harris wrote: >> Maybe I'll be wrong about learned indexes - who knows? But the burden >> of proof is not mine. I prefer to spend my time on things that I am >> reasonably confident will work out well ahead of time. > > > Agreed on all of your takes,

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Tue, Apr 20, 2021 at 3:45 PM Peter Geoghegan wrote: > On Tue, Apr 20, 2021 at 12:35 PM Chapman Flack > wrote: > > How would showing that to be true for data structure X be different from > > making a case for data structure X? > > You don't have to understand the theoretical basis of B-Tree

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Tue, Apr 20, 2021 at 12:35 PM Chapman Flack wrote: > How would showing that to be true for data structure X be different from > making a case for data structure X? You don't have to understand the theoretical basis of B-Tree indexes to see that they work well. In fact, it took at least a

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On 04/20/21 15:24, Peter Geoghegan wrote: > data structures that work well don't need anybody to make a case for them. > They simply work well for the task they were designed for. How would showing that to be true for data structure X be different from making a case for data structure X?

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Tue, Apr 20, 2021 at 11:18 AM Andrey Borodin wrote: > BTW take a look into PGM [0]. I'm slowly working on implementing it. > I think it is kind of straightforward to implement it as extension. > I've started from forking B-tree[1]. I've removed support of anything that is > not int4. > Then I

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

> 20 апр. 2021 г., в 22:56, Stefan Keller написал(а): > > Are there any advances in a learned index for PostgreSQL? BTW take a look into PGM [0]. I'm slowly working on implementing it. I think it is kind of straightforward to implement it as extension. I've started from forking

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

Dear Olegs, dear Nikolay, dear all Allow me to revive this thread: Are there any advances in a learned index for PostgreSQL? Background: I'm trying to benchmark those experimental indices. For this I did some bibliography work (see below). Fun fact: Not only Postgres people love high-proof

Re: Learned Index

On 12/12/2017 12:16 PM, Laurenz Albe wrote: I have read into the paper. This may be interesting or not, but the paper is very vague about its concepts and algorithms, so it's hard to tell. I'd say that the paper does not meet publication standards. For example, they say that their results

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On 12/12/2017 04:33 PM, Oleg Bartunov wrote: On Mon, Dec 11, 2017 at 11:11 PM, Nikolay Samokhvalov wrote: Very interesting read: https://arxiv.org/abs/1712.01208 HN discussion: https://news.ycombinator.com/item?id=15894896 Some of the comments (from Twitter

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

On Mon, Dec 11, 2017 at 11:11 PM, Nikolay Samokhvalov wrote: > Very interesting read: https://arxiv.org/abs/1712.01208 > > HN discussion: https://news.ycombinator.com/item?id=15894896 > > Some of the comments (from Twitter >

Re: Learned Index

Deepak Balasubramanyam wrote: > I came across this paper making a case for indices that use machine learning > to optimise search. > > https://arxiv.org/pdf/1712.01208.pdf > > The gist seems to be to use a linear regression model or feed a tensor flow > model when a more complicated