Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: > 1. Parallel workers help a lot when there is an expensive qualification > to evaluated, the more expensive the qualification the more better are > results.
I'd certainly hope so. ;) > 2. It works well for low selectivity quals and as the selectivity increases, > the benefit tends to go down due to additional tuple communication cost > between workers and master backend. I'm a bit sad to hear that the communication between workers and the master backend is already being a bottleneck. Now, that said, the box you're playing with looks to be pretty beefy and therefore the i/o subsystem might be particularly good, but generally speaking, it's a lot faster to move data in memory than it is to pull it off disk, and so I wouldn't expect the tuple communication between processes to really be the bottleneck... > 3. After certain point, increasing having more number of workers won't > help and rather have negative impact, refer Test-4. Yes, I see that too and it's also interesting- have you been able to identify why? What is the overhead (specifically) which is causing that? > I think as discussed previously we need to introduce 2 additional cost > variables (parallel_startup_cost, cpu_tuple_communication_cost) to > estimate the parallel seq scan cost so that when the tables are small > or selectivity is high, it should increase the cost of parallel plan. I agree that we need to figure out a way to cost out parallel plans, but I have doubts about these being the right way to do that. There has been quite a bit of literature regarding parallel execution and planning- have you had a chance to review anything along those lines? We certainly like to draw on previous experiences and analysis rather than trying to pave our own way. With these additional costs comes the consideration that we're looking for a wall-clock runtime proxy and therefore, while we need to add costs for parallel startup and tuple communication, we have to reduce the overall cost because of the parallelism or we'd never end up choosing a parallel plan. Is the thought to simply add up all the costs and then divide? Or perhaps to divide the cost of the actual plan but then add in the parallel startup cost and the tuple communication cost? Perhaps there has been prior discussion on these points but I'm thinking we need a README or similar which discusses all of this and includes any references out to academic papers or similar as appropriate. Thanks! Stephen
signature.asc
Description: Digital signature