On 31 October 2014 04:42, Amit Kapila <amit.kapil...@gmail.com> wrote:
>> In fact it would be more sensible to lock the toast table earlier. >> > > It might make some sense to lock the toast table earlier for this > particular case, but I don't think in general it will be feasible to lock > all the tables (including catalog tables) which might get used in the > overall operation before starting parallel operation. I believe it will > make doing parallel stuff difficult if we try to go via this route, as > we need to always do this for any operation we want to make parallel. > It will always have the risk that we might miss something and another > thing is it might not be feasible in all kind of cases. > > If I understand correctly, you are suggesting that to get the first version > of parallel operation out earlier, we should try to avoid all the stuff > (like group locking, ...), I advocate trying to avoid it, to see. The problems cited so far are not so extreme they cannot be easily got around, if you have a will to do so. I've just posted about an idea to reduce catalog locking. > have some restrictions on which kind of cases > Create Index can work, do some hackery in Create Index path to avoid > any kind of locking problems and do the parallel operation for External > Sort (which might avoid need for shared memory allocator) and then > call it a day and then do the things which we need for other parallel > operations as and when they are required. > I think this might be a good > approach in itself if somebody wants to target something to release > early, however in medium to short term when we want to provide > non-trivial parallel operations we have to eventually solve those problems > and delaying will only make the things worse. My (by now) long experience on Postgres has shown me that developing something now is a good route to take. By moving towards a solution at a reasonable pace you learn things which may affect the longer term viability of the project. New functionality in each release is useful. Big bang developments that don't deliver until the end have a habit of not delivering at the end either. "MVP" is a term frequently used in VCs for a reason. Lack of urgency in delivering a useful outcome from the project looks strange. If all you develop in this release is core infrastructure for parallelism and a custom scan API, it begins to look like there may not be an open source version delivered at all. How people make their income is up to them, but its a concern that's been raised my way before, so it seems reasonable to do so for others also. That is why we have spent much time explaining clearly the goals and licencing of BDR, for example. No need for big arguments; these are not allegations or implications etc.. > In short, I agree that there is a merit in your idea w.r.t getting the first > version of parallel operation out early, however if we see from medium > to long term, I feel solving group locking problem (or some other general > infrastructure what Robert mentioned upthread) can yield better results, > unless you feel that is not at all required for parallel operations. Is it genuinely required for most parallel operations? I think it's clear that none of us knows the answer. Sure, the general case needs it, but is the general case the same thing as the reasonably common case? Certainly, having a working prototype for something useful would help build the case for why these things are needed. It would certainly help the cause to have something working in this release. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers