The downside of conserve-sockets = false is that you are (essentially) back to a Thread|Socket / Request model (though Geode limits this system resource consumption to a degree by the use of Thread Pools in p2p distribution layer) and thus, you can run out of file descriptors (per newly opened Socket) pretty quickly if you are not careful.
conserve-sockets set to true limits the use of finite system resources why risking deadlocks (i.e. A -> B -> C -> A), which is also contingent on ACKS (and the infamous ReplyProcessor21; at least at 1 time, not sure if it is still in play, but probably!). conserve-sockets set to false uses more system resources but avoids deadlocks. If this change is made, I'd minimally make sure to document that users should adjust their (soft & hard) ulimits accordingly, based on use cases, load, etc. Personally, this has caused enough grief in the past (both ways, actually!) that I 'd say this is a major version change. -j ________________________________ From: Nabarun Nag <n...@vmware.com> Sent: Wednesday, November 18, 2020 6:09 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: [PROPOSAL] Change the default value of conserve-sockets to false +1 * As nearly all of the production use-cases need "conserve-sockets" to be set to false, I think we can aim for changing the default value to false for 1.14.0 release. * We can highlight this change in the release notes and emails. Regards, Nabarun ________________________________ From: Udo Kohlmeyer <u...@vmware.com> Sent: Wednesday, November 18, 2020 6:00 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: [PROPOSAL] Change the default value of conserve-sockets to false Hi there Donal, Thank you for raising this. It is not an uncommon request to change the default value of this field. This has been discussed many times in the past. I would LOVE to approve this change, but this would mean that users that don’t set this property might suddenly have this property changed. We are not sure that this would mean for these users. BUT That said, there have been very little (to none) complaints about the product stability when `conserve-sockets=false` are set. +1 - if we are allowed to make this change outside of a major version change. --Udo From: Donal Evans <doev...@vmware.com> Date: Thursday, November 19, 2020 at 12:04 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: [PROPOSAL] Change the default value of conserve-sockets to false Hi Geode dev, First, from the docs[1], a brief explanation of the purpose of the conserve-sockets property: "The conserve-sockets setting indicates whether application threads share sockets with other threads or use their own sockets for member communication. This setting has no effect on communication between a server and its clients, but it does control the server’s communication with its peers or a gateway sender’s communication with a gateway receiver." The current default value for the conserve-sockets property is true, which at first glance makes sense, since in an ideal world, existing sockets could be shared between threads and there would be no need to create and destroy new sockets for each process, which can be somewhat resource-intensive. However, in practice, there are several known issues with using the default setting of true. From the docs[1]: "For distributed regions, the put operation, and destroy and invalidate for regions and entries, can all be optimized with conserve-sockets set to false. For partitioned regions, setting conserve-sockets to false can improve general throughput. Note: When you have transactions operating on EMPTY, NORMAL or PARTITION regions, make sure that conserve-sockets is set to false to avoid distributed deadlocks." and[2]: "WAN deployments increase the messaging demands on a Geode system. To avoid hangs related to WAN messaging, always set `conserve-sockets=false` for Geode members that participate in a WAN deployment." Given that it is generally accepted as best practice to set conserve-sockets to false for almost all use cases of Geode beyond the most simple, it would make sense to also change the default value to false, to prevent people having to encounter a problem, search for the solution, then change the setting to what is almost always the "correct" value. I have done some experimenting to see what it would take to make this proposal a reality, and the changes required are very minimal, with only two existing DUnit tests that need to be modified to explicitly set the value of conserve-sockets that were previously relying on the default being true. Any feedback on this proposal would be very welcome, and if the response is positive, I can create a PR with the changes as soon as a decision is reached. Thanks, Donal [1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F113%2Fmanaging%2Fmonitor_tune%2Fperformance_controls_controlling_socket_use.html&data=04%7C01%7Cjblum%40vmware.com%7C87fe2b03a77045e6335408d88c3037b3%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637413486010510705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=dSxkBw04EOxr8vw1HP7u0oKMf%2FmNED6lVmaoV6FezGY%3D&reserved=0 [2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F113%2Fmanaging%2Fmonitor_tune%2Fsockets_and_gateways.html&data=04%7C01%7Cjblum%40vmware.com%7C87fe2b03a77045e6335408d88c3037b3%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637413486010510705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3tOT2lJPdd%2B1xBf4km5iQR6pVYIZVwcA6MelXyTbQ%2Bc%3D&reserved=0
