Hi Thomas,
I'm implementing the In-Memory index as per your suggestion. So far it's good. As of now, only one thing is unclear for me. How could i detach the dsa(dsa_detach() call) in backend (typically during backend quit). Of-course when process quits, all it's associated memory will be cleared/destroyed. But we have to decrement dsm reference counts as part of the potgres dsa framework implementation. which can not be done now. I have gone through the postgres source code. But unfortunately, i couldn't see anything in handling this case. Thanks a lot. Best Regards, -Mahi Teamwork divides the task and multiplies the success. ---- On Wed, 21 Jun 2017 13:13:28 +0530 Mahendranath Gurram <mahendran...@zohocorp.com> wrote ---- Hi Thomas, I like the whole idea. In fact, i understood this in your very first response itself. Only thing is every time i have to check for dsa_attached or not. I mean get_my_shared_state() is NULL or not. To avoid that check, i tried creating it in _PG_Init(postmaster process itself) all that stuff :( Anyways the solution you suggested will work for me. I'll live with that if check. Thanks a lot for all your help. I'll implement this and update the status. Cheers :) Thanks & Best Regards, -Mahi Teamwork divides the task and multiplies the success. ---- On Wed, 21 Jun 2017 12:02:30 +0530 Thomas Munro <thomas.mu...@enterprisedb.com> wrote ---- On Wed, Jun 21, 2017 at 5:27 PM, Mahendranath Gurram <mahendran...@zohocorp.com> wrote: > Initially i tried to design the same way. > I mean, i have created a background worker and created dsa in it. > I tried to attach/detach to the same dsa/dsm by all the backends(postgres > clients/connections) during backend(client/connection) init/destroy. > I didn't find any triggers or callbacks during backend init/close to > attach/detach the dsa/dsm. Hence, i took this approach. > If postgres have any such triggers/callbacks available, please let me know, > that is of great great help for me. > > Anyways now i understood, i have taken a wrong approach to use dsa. I'll try > to figure out any other way to build my in-memory index over postgres. You definitely can use DSA or DSM for this. As a matter of fact shared in-memory indexing was one of our target use cases. You just have to jump through a few hoops... Here's one approach: 1. Use _PG_init and the shmem hook to reserve a little bit of traditional shared memory and initialise it to zero. This will be used just to share the DSA handle, but you can't actually create the DSA area in postmaster. In other words, this little bit of shared memory is for "discovery", since it can be looked up by name from any backend. 2. In each backend that wants to use your new in-memory index system, you need to be able to attach or create the DSA area on-demand. Perhaps you could have a get_my_shared_state() function (insert better name) that uses a static local variable to hold a pointer to some state. If it's NULL, you know you need to create the state. That should happen only once in each backend, the first time through the function. In that case you need to create or attach to the DSA area as appropriate, which you should wrap in LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE)/LWLockRelease(AddinShmemInitLock) to serialise the code block. First, look up the bit of traditional shared memory to see if there is a DSA handle published in it already. If there is you can attach. If there isn't, you are the first so you need to create, and publish the handle for others to attach to. Remember whatever state you need to remember, such as the dsa_area, in static local variables so that all future calls to get_my_shared_state() in that backend will be fast. If you do it that way, then it doesn't matter whether it's background workers or foreground processes: whoever is first to call get_my_shared_state() will create the DSA area (and whatever else you want to do to set things up). All later callers will attach to the existing one. But each backend only ever has to enter the locked code path once, and from then on it's fast and there's no lock. Note that you could skip step 1 and not require a preload shared library. Then you'd be creating the 'discovery' shmem region on demand too! Just make sure it's in the locked region and pay attention to the 'found' output variable to decide whether you're first and need to initialise it. Skipping step 1 is very slightly against the rules though, because you'd be using a little piece of memory that you didn't tell the postmaster to reserve space for with RequestAddinShmemSpace. It's very small though, so you might decide that's OK... If you don't like the idea of creating that shmem stuff on demand, you could look into using session_preload_libraries as a way to get a 'backend init' hook. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
