On Thu, Sep 17, 2015 at 11:16 AM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > Is there some risk if we take too much large DSM segment? And what is max > size of DSM segment? When we use shm_mq, we don't need to solve limits.
I can't really think you are going to have a problem. How much data do you really intend to send back? Surely it's going to be <100kB. If you think it's not a problem to have a running query stop and send a gigabyte of data someplace anytime somebody asks, well, I don't think I agree. >> Also, if there are any bugs in the way the shm_mq is being used, >> they're likely to be quite rare and hard to find, because the vast >> majority of messages will probably be short enough to be sent in a >> single chunk, so whatever bugs may exist when the processes play >> ping-ping are unlikely to occur in practice except in unusual cases >> where the message being returned is very long. > > This is true for any functionality based on shm_mq - parallel seq scan, Parallel sequential scan is likely to put a lot more data through a shm_mq than you would for this. >> Second, using a shm_mq manipulates the state of the process latch. I >> don't think you can make the assumption that it's safe to reset the >> process latch at any and every place where we check for interrupts. >> For example, suppose the process is already using a shm_mq and the >> CHECK_FOR_INTERRUPTS() call inside that code then discovers that >> somebody has activated this mechanism and you now go try to send and >> receive from a new shm_mq. But even if that and every other >> CHECK_FOR_INTERRUPTS() in the code can tolerate a process latch reset >> today, it's a new coding rule that could easily trip people up in the >> future. > > It is valid, and probably most important. But if we introduce own mechanism, > we will play with process latch too (although we can use LWlocks) With the design I proposed, there is zero need to touch the process latch, which is good, because I'm pretty sure that is going to be a problem. I don't think there is any need to use LWLocks here either. When you get a request for data, you can just publish a DSM segment with the data and that's it. Why do you need anything more? You could set the requestor's latch if it's convenient; that wouldn't be a problem. But the process supplying the data can't end up in a different state than it was before supplying that data, or stuff WILL break. >> Using a shm_mq is appropriate when the amount of data that needs to be >> transmitted might be very large - too large to just allocate a buffer >> for the whole thing - or when the amount of data can't be predicted >> before memory is allocated. But there is obviously no rule that a >> shm_mq should be used any time we have "data exchange between >> processes"; we have lots of shared-memory based IPC already and have >> for many years, and shm_mq is newer than the vast majority of that >> code. > > I am little bit disappointed - I hoped so shm_mq can be used as generic > interprocess mechanism - that will ensure all corner cases for work with > shared memory. I understand to shm_mq is new, and nobody used it, but this > risk is better than invent wheels again and again. shm_mq is useful, but if you insist on using a complicated tool when a simple one is plenty sufficient, you may not get the results you're hoping for. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
