Re: Compressed TOAST Slicing

2019-04-09 Thread Andrey Borodin
Hi! > 12 марта 2019 г., в 10:22, Andrey Borodin написал(а): > > 3. And I'd use memmove despite the comment why we do not do that. It is > SSE-optimized and cache-optimized nowadays. So, I've pushed idea a little bit and showed that decompress byte-copy cycle to Vladi

Re: Google Summer of Code: question about GiST API advancement project

2019-04-05 Thread Andrey Borodin
part, it is MR subsystem. Best regards, Andrey Borodin.

Re: Google Summer of Code: question about GiST API advancement project

2019-04-05 Thread Andrey Borodin
happens with geo data, all points have same height above ocean) if always have zero volume of MBB. BTW look at PostGIS, they are main GiST users, so indexing their data is important. Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-04-04 Thread Andrey Borodin
Hi! > 4 апр. 2019 г., в 20:15, Heikki Linnakangas написал(а): > > On 25/03/2019 15:20, Heikki Linnakangas wrote: >> On 24/03/2019 18:50, Andrey Borodin wrote: >>> I was working on new version of gist check in amcheck and understand one >>> more thing: >>

Re: [GSoC] application ideas

2019-04-04 Thread Andrey Borodin
Hi! We are discussing GSoC details offlist, but I'll put some recommendations on your proposal to the list. > 3 апр. 2019 г., в 2:53, pantilimonov misha написал(а): > > Andrey, thank you for your reply. > >> 24.03.2019, 12:12, "Andrey Borodin" : >>

Re: Google Summer of Code: question about GiST API advancement project

2019-03-31 Thread Andrey Borodin
ng subtree for insertion. R*-tree algorithms are heuristic only to be faster. Best regards, Andrey Borodin. [0] https://arxiv.org/pdf/1712.01208.pdf

Re: amcheck verification for GiST

2019-03-28 Thread Andrey Borodin
> 29 марта 2019 г., в 5:35, Peter Geoghegan написал(а): > > On Thu, Mar 28, 2019 at 10:08 AM Andrey Borodin wrote: >>>> Is this really needed? Isn't the ShareLock on the index sufficient? If so, >>>> why? >>> There may be concurrent

Re: amcheck verification for GiST

2019-03-28 Thread Andrey Borodin
> 28 марта 2019 г., в 18:35, Andrey Borodin написал(а): >> >> Is this really needed? Isn't the ShareLock on the index sufficient? If so, >> why? > There may be concurrent inserts? In GiST they can reorder items on page. Looks like I've tried to cope with s

Re: amcheck verification for GiST

2019-03-28 Thread Andrey Borodin
Thanks for looking into this! > 27 марта 2019 г., в 22:29, Heikki Linnakangas написал(а): > > On 27/03/2019 11:51, Andrey Borodin wrote: >> Hi! >> Here's new version of GiST amcheck, which takes into account recently >> committed GiST VACUUM. >> It tests

Re: amcheck verification for GiST

2019-03-27 Thread Andrey Borodin
Hi! Here's new version of GiST amcheck, which takes into account recently committed GiST VACUUM. It tests that deleted pages do not contain any data. Also, Heikki's fix applied (wrong OffsetNumberNext(i) replaced by OffsetNumberNext(o)). Thanks! Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-03-24 Thread Andrey Borodin
eems like same applies to GIN...) Best regards, Andrey Borodin.

Re: [GSoC] application ideas

2019-03-24 Thread Andrey Borodin
he idea of more efficient BufferTag->Page data structure. I'm not sure cache locality is a real problem there, but I believe this idea deserves giving it a shot. I'd happily review your proposal and co-mentor project, if it will be chosen for GSoC. Also, plz check some work of my stud

Re: GiST VACUUM

2019-03-22 Thread Andrey Borodin
.g. 4 > sub-pages, with a "bounding box" key for each sub-page, to speed up search. BTW, I already have intra-page indexing patch. But now it obviously need a rebase :) Along with gist amcheck patch. Thanks! Best regards, Andrey Borodin.

Re: Special role for subscriptions

2019-03-22 Thread Andrey Borodin
F, the topic of subscription security relaxation really worth efforts. Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-03-22 Thread Andrey Borodin
> allow reusing deleted pages immediately. Unfortunately that would require > adding a new field to the gist page header/footer, which requires upgrade > work :-(. Maybe one day, we'll bite the bullet. Something to keep in mind, if > we have to change the page format anyway, for some reason. Yeah, the same day we will get rid of invalid tuples. I can make a patch for v13. Actually, I have a lot of patches that I want in GiST in v13. Or v14. Best regards, Andrey Borodin.

Re: Special role for subscriptions

2019-03-21 Thread Andrey Borodin
n with some security checks 3. We have complete list of possible security checks 4. We have code that implements most of these checks (I believe pg_subscription_role_v2.patch is enough, but we can tighten checks a little more) Do we have any objection on these points? If not, it is RFC, it should not be returned. Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-03-21 Thread Andrey Borodin
st thing that can happen: it will return same tid twice. But it is doing bitmap scan, you cannot return same bit twice... Eventually, hash, spgist and others will have same thing too. Best regards, Andrey Borodin.

Re: Special role for subscriptions

2019-03-20 Thread Andrey Borodin
> 21 марта 2019 г., в 8:56, Michael Paquier написал(а): > > On Wed, Mar 20, 2019 at 11:58:04PM +0800, Andrey Borodin wrote: >>> 20 марта 2019 г., в 21:46, Robert Haas написал(а): >>> I think we should view this permission as "you can create >>> subs

Re: Special role for subscriptions

2019-03-20 Thread Andrey Borodin
g. to the managed cloud with vanilla postgres). But the role effectively allows inserts to any table, this can be escalated to superuser. What is the best way to deal with it? Best regards, Andrey Borodin.

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-19 Thread Andrey Borodin
ost of this printing usages were there before your patch. I'm marking the patch as RFC, since I have no more notices and patch really looks good. Best regards, Andrey Borodin.

Re: Sparse bit set data structure

2019-03-19 Thread Andrey Borodin
added while iteration is in progress. You check for violation of these limitation in code, but there is not tests for this checks. Should we add these tests? Best regards, Andrey Borodin.

Re: Special role for subscriptions

2019-03-19 Thread Andrey Borodin
ow, > should we check it? Currently, user with pg_subscription_users can create subscription into any system table, can't they? We certainly need to change it to more secure way. Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-03-15 Thread Andrey Borodin
Hi! > 11 марта 2019 г., в 20:03, Heikki Linnakangas написал(а): > > On 10/03/2019 18:40, Andrey Borodin wrote: >> Here's new version of the patch for actual page deletion. >> Changes: >> 1. Fixed possible concurrency issue: >> We were locking child pa

Re: Special role for subscriptions

2019-03-14 Thread Andrey Borodin
5. Condition 2 can be replaced\extended by "have privileges for some server remote". Which combination of authorization rules do we want? IMHO 1,2,4 is sufficient. Best regards, Andrey Borodin.

Re: Sparse bit set data structure

2019-03-13 Thread Andrey Borodin
Hi! > 14 марта 2019 г., в 0:18, Heikki Linnakangas написал(а): > <0001-Add-SparseBitset-to-hold-a-large-set-of-64-bit-ints-.patch><0002-Andrey-Borodin-s-test_blockset-tool-adapted-for-Spar.patch> That is very interesting idea. Basically, B-tree and radix tree is a tradeoff

Re: Compressed TOAST Slicing

2019-03-13 Thread Andrey Borodin
break; /* do not clobber memory */ } } else Best regards, Andrey Borodin.

Re: GSOC Application

2019-03-13 Thread Andrey Borodin
PT, PYTHON and even SQL. Though, discussed project relates to SQL to some degree. There are some ideas requiring Python, I'd like to encourage you look closer to them. Best regards, Andrey Borodin.

Re: Compressed TOAST Slicing

2019-03-12 Thread Andrey Borodin
amp;& sp != srcend) return -1; and that's it. || defends from data corruption and some kind of programming mistakes, but actually that's not the purpose of data compression. Best regards, Andrey Borodin.

Re: Compressed TOAST Slicing

2019-03-11 Thread Andrey Borodin
t entirely? 3. And I'd use memmove despite the comment why we do not do that. It is SSE-optimized and cache-optimized nowadays. But this in not point of this patch, so let's discard this point. Best regards, Andrey Borodin.

Re: Covering GiST indexes

2019-03-10 Thread Andrey Borodin
Hi! > 10 марта 2019 г., в 13:39, Alexander Korotkov > написал(а): > > Pushed with some more small changes in docs. That's great, thank you! Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-03-10 Thread Andrey Borodin
> 5 марта 2019 г., в 18:21, Heikki Linnakangas написал(а): > > On 05/03/2019 02:26, Andrey Borodin wrote: >> >> That's cool! I'll work on 2nd step of these patchset to make >> blockset data structure prettier and less hacky. > > Committed the firs

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-03-06 Thread Andrey Borodin
n MAXPGPATH and MAX_RESTORE_COMMAND? Besides this, patch looks fine to me. Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-03-04 Thread Andrey Borodin
this as > Ready for Committer in the commitfest app, and will try to commit it in the > next couple of days. That's cool! I'll work on 2nd step of these patchset to make blockset data structure prettier and less hacky. Best regards, Andrey Borodin.

Re: GSoC 2019

2019-03-02 Thread Andrey Borodin
r WAL-G related discussions. We have a WAL-G Slack channel https://postgresteam.slack.com/messages/CA25P48P2 . To get there you can use this invite app https://postgres-slack.herokuapp.com/ There you can ask whatever you want to know about WAL-G or your GSOC proposal. Thanks for your interest in our project! Best regards, Andrey Borodin.

Re: amcheck verification for GiST

2019-02-24 Thread Andrey Borodin
ept my patch on commitfest). I suspect that there may be deleted pages from previous versions. But encountering this in a search should not be a problem. Thus, I copy behavior from index scan: do not complain about deleted pages. > * Do we need to worry about F_HAS_GARBAGE pages? Why or why not? This flag is for microvacuum and only hints that there may be some vacuumable tuples on page. It is used during check for neccesary split to allocate some space. Thanks! Best regards, Andrey Borodin. 0001-GiST-verification-function-for-amcheck-v5.patch Description: Binary data

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-02-17 Thread Andrey Borodin
moving to the cloud DB where user has no superuser privileges. Best regards, Andrey Borodin.

Re: amcheck verification for GiST

2019-02-17 Thread Andrey Borodin
Hi Peter! Sorry for the delay. Here's new version. > 1 февр. 2019 г., в 4:58, Peter Geoghegan написал(а): > > On Tue, Jan 1, 2019 at 2:11 AM Andrey Borodin wrote: >>> This isn't consistent with what you do within verify_nbtree.c, which >>> deliberately a

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-02-14 Thread Andrey Borodin
rop this lines from docs: > The subscription apply process will run in the local database with the > privileges of a superuser. Thanks! Best regards, Andrey Borodin.

[Patch][WiP] Tweaked LRU for shared buffers

2019-02-13 Thread Andrey Borodin
Hi, hackers! We have held education project at Sirius edu center (Sochi, Russia) with mentors from Yandex. The group of 5 students was working on improving the shared buffers eviction algorithm: Andrey Chausov, Yuriy Skakovskiy, Ivan Edigaryev, Arslan Gumerov, Daria Filipetskaya. I’ve been a me

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-02-10 Thread Andrey Borodin
on test we only subtract test (fail for nonsuperuser). Can we add some test there too? All these are merely cosmetical issues. I believe after addressing these we can switch patch to Ready For Committer. Best regards, Andrey Borodin.

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-02-09 Thread Andrey Borodin
ent in the target directory\n"), I'm still not sure guc-file refactoring you made is architecturally correct, I do not feel that my expertise is enough to judge, but everything works. Besides this, I think you can switch patch to "Ready for committer". check-world is passing on

Re: Covering GiST indexes

2019-01-29 Thread Andrey Borodin
> 29 янв. 2019 г., в 23:43, Andreas Karlsson написал(а): > > On 29/01/2019 18.45, Andrey Borodin wrote: >> Also, I've unified gist and r-tree syntax tests for INCLUDE. > > Why not just keep it simple? The purpose is not to test the GiST indexes, for > that we

Re: Covering GiST indexes

2019-01-29 Thread Andrey Borodin
x27;ve unified gist and r-tree syntax tests for INCLUDE. Best regards, Andrey Borodin. 0001-Covering-GiST-v8.patch Description: Binary data

Re: Covering GiST indexes

2019-01-28 Thread Andrey Borodin
> 29 янв. 2019 г., в 7:32, Andreas Karlsson написал(а): > > On 1/28/19 7:26 PM, Andrey Borodin wrote: >>> * I am no fan of the tupdesc vs truncTupdesc separation and think that it >>> is a potential hazard, but I do not have any better suggestion right now. >>

Re: Covering GiST indexes

2019-01-28 Thread Andrey Borodin
ays support > index-only scans." rather than "Included attributes can be returned." in the > comment for gistcanreturn(). > Fixed, but slightly reworded. > * Why the random noise in the diff below? I think using "(c3) INCLUDE (c4)" > for both gist an

Re: O_DIRECT for relations and SLRUs (Prototype)

2019-01-15 Thread Andrey Borodin
gt; aligned with the OS pages, which should be 4k for Linux by the way, > and not set to BLCKSZ, but for WAL's O_DIRECT we don't really care > much with such details. Is it possible to avoid those memcopy's by aligning available buffers instead? I couldn't understand this from

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-01-14 Thread Andrey Borodin
n preset subscription" Should it be "table does not present in subscription"? Besides this, patch looks good to me. Thanks for working on this! Best regards, Andrey Borodin.

Re: O_DIRECT for relations and SLRUs (Prototype)

2019-01-13 Thread Andrey Borodin
e is done wrong" due to really bad performance. Best regards, Andrey Borodin.

Re: O_DIRECT for relations and SLRUs (Prototype)

2019-01-12 Thread Andrey Borodin
hings get significantly slower. Cool! I've just gathered a group of students to task them with experimenting with shared buffer eviction algorithms during their February internship at Yandex-Sirius edu project. Your patch seems very handy for benchmarks in this area. Thanks! Best regards, Andrey Borodin.

Re: GSoC 2019

2019-01-10 Thread Andrey Borodin
o single project. Vladimir had done 2018's WAL-G project during Yandex internship, so I'll remove this project from page. Best regards, Andrey Borodin.

Re: commitfest: When are you assigned patches to review?

2019-01-08 Thread Andrey Borodin
Hi everyone! > 8 янв. 2019 г., в 14:14, Fabien COELHO написал(а): > > The process is that *you* choose the patches to review and register as such > for the patch on the CF app. By the way, is it ok to negotiate review exchange? Best regards, Andrey Borodin.

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2019-01-07 Thread Andrey Borodin
so, let's comment about it. 3. RestoreArchivedFile() checks for signals, is it done by pg_rewind elsewhere? 4. No documentation is updated 5. -R takes precedence over -r without notes. Shouldn't we complain? Or may be we should take one from config, iif nothing found use -R? Thanks! Best regards, Andrey Borodin.

Re: GiST VACUUM

2019-01-04 Thread Andrey Borodin
3 янв. 2019 г., в 23:47, Andrey Borodin написал(а): > >> * Bitmapset stores 32 bit signed integers, but BlockNumber is unsigned. So >> this will fail with an index larger than 2^31 blocks. That's perhaps >> academical, I don't think anyone will try to create

Re: GiST VACUUM

2019-01-03 Thread Andrey Borodin
ent to GistPageSetDeleteXid() * In this check > if (GistPageIsDeleted(page) && > TransactionIdPrecedes(GistPageGetDeleteXid(page), RecentGlobalXmin)) I've switched using RecentGlobalDataXmin to RecentGlobalXmin, because we have done so in similar mechanics for GIN (for uniformity with B-tree). Thanks for working on this! Best regards, Andrey Borodin. 0002-Delete-pages-during-GiST-VACUUM-v19.patch Description: Binary data 0001-Physical-GiST-scan-in-VACUUM-v19.patch Description: Binary data

Re: amcheck verification for GiST

2019-01-01 Thread Andrey Borodin
Hi, Peter! Thank you for the review! > 7 дек. 2018 г., в 3:59, Peter Geoghegan написал(а): > > On Sun, Sep 23, 2018 at 10:12 PM Andrey Borodin wrote: > * You do this: > >> +/* Check of an internal page. Hold locks on two pages at a time >> (parent+child). */ >

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2018-12-24 Thread Andrey Borodin
se, avoid top-posting (quoting whole message under your reply), this makes harder to read archives at postgresql.org And also please send patches with version number to distinguish old and new versions. Best regards, Andrey Borodin.

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-13 Thread Andrey Borodin
ed Hat 4.4.7-23) That's the same variable, one place is definition while other is potential misuse. Seems like these 2 lines [0] + if (BufferIsValid(lbuffer)) + UnlockReleaseBuffer(lbuffer); are superfluous: lbuffer is UnlockReleased earlier. Best regards, Andrey Borodin. [0] https://github.com/postgres/postgres/commit/19cf52e6cc33b9e126775f28269b6ddf7e30#diff-ed6446a8993c76d2884ec6413e49a6b6R757

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-12 Thread Andrey Borodin
ecks this condition both before putting page into FSM and > after getting page from FSM. You're checking only after getting page > from FSM. Approach of B-tree looks better for me. It's seems more > consistent when FSM pages are really free for usage. Fixed. Best regards, Andrey

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-10 Thread Andrey Borodin
ePage is never set. Sorry, I've messed it again. Forgot to include ginvacuum.c changes. Here it is. Best regards, Andrey Borodin. 0001-Stamp-deleted-GIN-page-with-xid-v2.patch Description: Binary data

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-10 Thread Andrey Borodin
> 10 дек. 2018 г., в 18:56, Andrey Borodin написал(а): > > PFA this part. In thread about GiST vacuum Heikki was not very happy with > reusing PageHeader->pd_prune_xid for this. But we somewhat concluded that > this might be OK. > Anyway, there's whole page, we can

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-10 Thread Andrey Borodin
avior? It seems to me that locking order must be left->deleted->parent. Best regards, Andrey Borodin.

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-08 Thread Andrey Borodin
he page before TransactionIdPrecedes(GinPageGetDeleteXid(page), RecentGlobalDataXmin). Should we leave alone this bug for future fixes to keep current fix noninvasive? Best regards, Andrey Borodin.

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-07 Thread Andrey Borodin
tree for cleanup. PFA diff with following changes: 1. Always take root cleanup lock before deleting pages 2. Check for concurrent splits after scanning page Please note, that neither applying this diff nor reverting 218f51584d5 will solve bug of page delete redo lock on standby. Best regards, Andrey Borodin. gin_root_lock.diff Description: Binary data

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2018-12-02 Thread Andrey Borodin
о быть суперпользователем"? Where do we insert FOR ALL TABLES? 4. How does default behavior differ from FOR ALL TABLES? 5. Can we alter subscription FOR ALL TABLES? Drop some tables out of the subscription? Best regards, Andrey Borodin.

Re: [HACKERS] Can ICU be used for a database's default sort order?

2018-12-02 Thread Andrey Borodin
Hi, Dmitry, Daniel! > 2 дек. 2018 г., в 17:22, Dmitry Dolgov <9erthali...@gmail.com> написал(а): > >> There were reviews: Andrey Borodin raised issues with the patch in >> [2], I spent some time trying it and asked questions about the design >> in [3], but no one

Re: Covering GiST indexes

2018-11-26 Thread Andrey Borodin
lease rebase it one more time? Here's rebased version. Thanks! Best regards, Andrey Borodin. 0001-Covering-GiST-v5.patch Description: Binary data

Re: B-tree cache prefetches

2018-11-26 Thread Andrey Borodin
seems to be winner. How can I improve experimental evaluation of these layout strategies? Other thoughts? I'd be happy to hear any comment on this. Best regards, Andrey Borodin. 0001-Implement-different-B-tree-page-layouts.patch Description: Binary data

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-11-23 Thread Andrey Borodin
Hi! > 15 нояб. 2018 г., в 17:30, Andrey Borodin написал(а): > > I think I can compose patch for consideration. I found no practical way to fix concept of "subtree-lock". Pre-locking all left siblings for cleanup would suffice, but does not look very practical. GIN Post

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-11-15 Thread Andrey Borodin
Hi! Mail.App somehow borken my reponse with , which is far beyond of my understanding of what is plain text, that's why I'll quote all my previous message here. Hope this mail client is OK. 14.11.2018, 23:13, "Andrey Borodin" : > Hi everyone! > > I didn'

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-11-14 Thread Andrey Borodin
be change from "dbuffer->pbuffer->lbuffer" to "lbuffer->dbuffer->pbuffer" . Is this right?This looks correct to me.Best regards, Andrey Borodin.

Re: Connection limit doesn't work for superuser

2018-11-07 Thread Andrey Borodin
restrictions within the database. Do we consider connection limit "access restriction"? Superuser can avoid setting his connection limit if he do not need it. Best regards, Andrey Borodin.

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-28 Thread Andrey Borodin
Hi, Alexey! > 25 окт. 2018 г., в 17:37, Alexey Kondratov > написал(а): Will you add this patch to CF? I'm going to review it. Best regards, Andrey Borodin

Re: GiST VACUUM

2018-10-28 Thread Andrey Borodin
Hi everyone! > 2 окт. 2018 г., в 6:14, Michael Paquier написал(а): > Andrey, your latest patch does not apply. I am moving this to the next > CF, waiting for your input. I'm doing preps for CF. Here's rebased version. Best regards, Andrey Borodin. 0002-Delete-pages-dur

Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line

2018-10-22 Thread Andrey Borodin
else > came to mind. It is also possible to leave the only one option -- passing > restore_command as command line argument. > I think it is better to load restore_command from recovery.conf. I didn't actually try patch yet, but the idea seems interesting. Will you add it to the commitfest? Best regards, Andrey Borodin.

Re: B-tree cache prefetches

2018-10-14 Thread Andrey Borodin
> 15 окт. 2018 г., в 2:38, Thomas Munro > написал(а): > > On Mon, Oct 15, 2018 at 12:03 AM Andrey Borodin wrote: >> 31 авг. 2018 г., в 2:40, Thomas Munro >> написал(а): >> A related topic is the cache-unfriendliness of traditional binary >> search

Re: B-tree cache prefetches

2018-10-14 Thread Andrey Borodin
-tree binsearch: thier data items are small and do not represent reference to actual data. Eytzinger exploits the fact that two posiible future keys share same cache line. But it is hard to provide, given we place items at quite random place of the page. Best regards, Andrey Borodin.

Re: Experimenting with hash join prefetch

2018-10-14 Thread Andrey Borodin
Hi, Thomas! > 14 окт. 2018 г., в 9:18, Thomas Munro > написал(а): > > + /* Prefetch the bucket for the next key */ > + uint32 next_hash = hash_uint32(DatumGetInt32(keyval) + 1); > + uint32 next_bucket = next_hash % hashtable->nbuckets; > +

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-10-02 Thread Andrey Borodin
ost no > feedback since 20 Mar after patch design was changed :) > But seriously - I still working on it Let's move this to CF 2018-11? Obviously, it is WiP, but it seems that patch is being discussed, author cares about it. Best regards, Andrey Borodin.

Re: Global snapshots

2018-09-24 Thread Andrey Borodin
tically is kind of dangerous... Also, currently hanging 2pc transaction can cause a lot of headache for DBA. Can we have some kind of protection for the case when one node is gone permanently during transaction? Thanks! Best regards, Andrey Borodin.

Re: amcheck verification for GiST

2018-09-23 Thread Andrey Borodin
Hi! > 24 сент. 2018 г., в 8:29, Thomas Munro > написал(а): > > On Sun, Sep 23, 2018 at 10:15 PM Andrey Borodin wrote: >> Here's the patch with amcheck functionality for GiST. > > Hi Andrey, > > Windows doesn't like it[1]: Thanks, Thomas! Yes,

amcheck verification for GiST

2018-09-23 Thread Andrey Borodin
tree invariants due to concurrency reasons some concurrent splits will be visible as temporary balance violations. Are there any other invariants that we can check? I'd be happy to hear any thought about this. Best regards, Andrey Borodin. 0001-GiST-verification-function-for-amcheck.patch D

Re: [HACKERS] [PATCH] kNN for SP-GiST

2018-09-17 Thread Andrey Borodin
Hi! > 17 сент. 2018 г., в 2:03, Alexander Korotkov > написал(а): > > Also, it appears to me that it's OK to be a single patch +1, ISTM that these 6 patches represent atomic unit of work. Best regards, Andrey Borodin.

Re: A strange GiST error message or fillfactor of GiST build

2018-09-05 Thread Andrey Borodin
> 5 сент. 2018 г., в 13:29, Kyotaro HORIGUCHI > написал(а): > > We don't preserve it for Andrey's use case. Just my 2 cents: that was a hacky use case for development reasons. I think that removing fillfactor is good idea and your latest patch looks good from my POV.

Re: A strange GiST error message or fillfactor of GiST build

2018-09-03 Thread Andrey Borodin
ome > experiments first, and then propose a patch. FWIW fillfactor can be a cheap emulator for intrapage indexing, when you have like a lot of RAM. Though I didn't tested that. Also I believe proper intrapage indexing is better :) Best regards, Andrey Borodin.

B-tree cache prefetches

2018-08-30 Thread Andrey Borodin
lect (random()*100)::int from generate_series(1,1e7); it brings something like 2-4% of performance improvement on my laptop. Is there a reason why we do not use __builtin_prefetch? Have anyone tried to use cache prefetching? Best regards, Andrey Borodin.

Re: A strange GiST error message or fillfactor of GiST build

2018-08-30 Thread Andrey Borodin
Hello! > 30 авг. 2018 г., в 2:42, Kyotaro HORIGUCHI > написал(а): > > At Wed, 29 Aug 2018 10:42:59 -0300, Andrey Borodin > wrote in <6fbe12b2-4f59-4db9-bde9-62c880118...@yandex-team.ru> >> >> We are passing freespace everywhere. Also, we pass GistInsertS

Re: A strange GiST error message or fillfactor of GiST build

2018-08-29 Thread Andrey Borodin
t; After the attached patch applied, the above messages becomes as > follows. (And index can be built being a bit sparse by fill > factor.) We are passing freespace everywhere. Also, we pass GistInsertState, and GistState. Maybe let's put GistState into GistInsertState, GistState already has free space, and pass just GistInsertState everywhere? Best regards, Andrey Borodin.

Re: Would it be possible to have parallel archiving?

2018-08-28 Thread Andrey Borodin
thod will stay safe forever. But now it is. Best regards, Andrey Borodin.

Re: Would it be possible to have parallel archiving?

2018-08-28 Thread Andrey Borodin
> 28 авг. 2018 г., в 17:07, Stephen Frost написал(а): > > Greetings, > > * Andrey Borodin (x4...@yandex-team.ru) wrote: >>> 28 авг. 2018 г., в 14:08, Stephen Frost написал(а): >>> * David Steele (da...@pgmasters.net <mailto:da...@pgmasters.net>) wrote:

Re: Would it be possible to have parallel archiving?

2018-08-28 Thread Andrey Borodin
e. Indeed, it was very hard to test. Also, this makes impossible to use two archiving system simultaneously for transit period. Best regards, Andrey Borodin.

Re: Dimension limit in contrib/cube (dump/restore hazard?)

2018-08-28 Thread Andrey Borodin
> 28 авг. 2018 г., в 14:18, Alexander Korotkov > написал(а): > > OK, but I think cube_c_f8() and cube_c_f8_f8() also need to be > revised. Also, I think this behavior should be covered by regression > tests. True. Also there's one case in cube_subset. Best

Re: Dimension limit in contrib/cube (dump/restore hazard?)

2018-08-28 Thread Andrey Borodin
de(ERRCODE_ARRAY_ELEMENT_ERROR), +errmsg("A cube cannot have more than %d dimensions.", + CUBE_MAX_DIM))); dur = ARRPTR(ur); Best regards, Andrey Borodin.

Re: Proposal: SLRU to Buffer Cache

2018-08-22 Thread Andrey Borodin
ng something, but why this should not be in access methods? You can extend AM to control it's segment size and ability to truncate unneeded pages. This may to be useful, for example, in LSM tree implementation or something similar. Best regards, Andrey Borodin.

Re: GiST VACUUM

2018-08-06 Thread Andrey Borodin
Hi! PFA v16. > 5 авг. 2018 г., в 21:45, Andrey Borodin написал(а): >> 5 авг. 2018 г., в 16:18, Heikki Linnakangas написал(а): >> >> Hmm. A ListCell is 16 bytes, plus the AllocChunk header, 16 bytes. 32 >> bytes per internal page in total, while a bitmap consumes on

Re: GiST VACUUM

2018-08-05 Thread Andrey Borodin
Hi! > 5 авг. 2018 г., в 16:18, Heikki Linnakangas написал(а): > > On 31/07/18 23:06, Andrey Borodin wrote: >>> On a typical GiST index, what's the ratio of leaf vs. internal >>> pages? Perhaps an array would indeed be better. > > >> Typical GiST h

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andrey Borodin
out preventing writing bad writes. But adding LSNs, and whole regular PageHeader is quite easy in this patch. Do you think we should really go that way? Putting SLRUs into usual shared buffers and WAL-logging looks like a good idea, but a lot of work. Best regards, Andrey Borodin.

Re: GiST VACUUM

2018-07-31 Thread Andrey Borodin
Hi! Thanks for looking into the patch! > 30 июля 2018 г., в 18:39, Heikki Linnakangas написал(а): > > On 29/07/18 14:47, Andrey Borodin wrote: >> Fixed both problems. PFA v14. > > Thanks, took a really quick look at this. > > The text being added to README is outdat

Re: Covering GiST indexes

2018-07-29 Thread Andrey Borodin
Thanks, Thomas! > 30 июля 2018 г., в 3:58, Thomas Munro > написал(а): > > On Sun, Jul 29, 2018 at 10:50 PM, Andrey Borodin wrote: >> Thanks! The problem appeared with commit 701fd0b [0] which dropped >> validation rules checked in failed test. Here's the patch

Re: GiST VACUUM

2018-07-29 Thread Andrey Borodin
Hi! Thank you! > 29 июля 2018 г., в 14:04, Thomas Munro > написал(а): > > On Tue, Jul 24, 2018 at 6:04 AM, Andrey Borodin wrote: >>> 21 июля 2018 г., в 17:11, Andrey Borodin написал(а): >>> <0001-Physical-GiST-scan-in-VACUUM-v13.patch> >> >>

Re: Covering GiST indexes

2018-07-29 Thread Andrey Borodin
Hi, Thomas!29 июля 2018 г., в 14:28, Thomas Munro <thomas.mu...@enterprisedb.com> написал(а):On Fri, Jul 6, 2018 at 5:27 AM, Andrey Borodin <x4...@yandex-team.ru> wrote:Here is v3 version of the patch. I've fixed some comments and added some words to docs.Hi again Andrey,C

<    1   2   3   4   5   6   7   >