Re: [DISCUSS] Versionless Spark Programming Guide Proposal

2024-06-05 Thread Matthew Powers
I am a huge fan of the Apache Spark docs and I regularly look at the analytics on this page to see how well they are doing. Great work to everyone that's contributed to the

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

2024-06-05 Thread Neil Ramaswamy
Thanks all for the responses. Let me try to address everything. > the programming guides are also different between versions since features are being added, configs are being added/ removed/ changed, defaults are being changed etc. I agree that this is the case. But I think it's fine to mention

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

2024-06-05 Thread Wenchen Fan
I agree with the idea of a versionless programming guide. But one thing we need to make sure of is we give clear messages for things that are only available in a new version. My proposal is: 1. keep the old versions' programming guide unchanged. For example, people can still access

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

2024-06-05 Thread Martin Andersson
While I have no practical knowledge of how documentation is maintained in the spark project, I must agree with Nimrod. For users on older versions, having a programming guide that refers to features or API methods that does not exist in that version is confusing and detrimental. Surely there

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

2024-06-05 Thread Nimrod Ofek
Hi Neil, While you wrote you don't mean the api docs (of course), the programming guides are also different between versions since features are being added, configs are being added/ removed/ changed, defaults are being changed etc. I know of "backport hell" - which is why I wrote that once a