Re: [DISCUSS] Show Python code examples first in Spark documentation

Yes, I definitely agree and +1 to the proposal (FWIW). I was looking at Dongjoon's comments which made a lot of sense to me and trying to come up with an approach that provides smooth segway to python as first tab later on. But this is mostly guess work as I do not personally know the actual user

Re: [DISCUSS] Show Python code examples first in Spark documentation

That sounds good to have that especially given that it will allow more flexibility to the users. But I think that's slightly orthogonal to this proposal since this proposal is more about the default (before users take an action). On Fri, 24 Feb 2023 at 15:35, Santosh Pingale wrote: > Very

Re: [DISCUSS] Show Python code examples first in Spark documentation

Very interesting and user focused discussion, thanks for the proposal. Would it be better if we rather let users set the preference about the language they want to see first in the code examples? This preference can be easily stored on the browser side and used to decide ordering. This is inline

Re: [DISCUSS] Show Python code examples first in Spark documentation

I think this needs to be consistently done on all relevant pages and my intent is to do that work in time for when it is first released. I started with the "Spark SQL, DataFrames and Datasets Guide" page to break it up into multiple, scoped PRs. I should have made that clear before. I think it's

Re: [DISCUSS] Show Python code examples first in Spark documentation

Thank you all. Yes, attracting more Python users and being more Python user-friendly is always good. Basically, SPARK-42493 is proposing to introduce intentional inconsistency to Apache Spark documentation. The inconsistency from SPARK-42493 might give Python users the following questions

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

Yes we should fix. I will take a look On Thu, 23 Feb 2023 at 07:32, Jonathan Kelly wrote: > Thanks! I was wondering about that ClientE2ETestSuite failure today, so > I'm glad to know that it's also being experienced by others. > > On a similar note, I am experiencing the following error when

Re: [DISCUSS] Show Python code examples first in Spark documentation

Thanks a lot for the questions and comments/feedback! To address your questions Dongjoon, I do not intend for these updates to the documentation to be tied to the potential changes/suggestions you ask about. In other words, this proposal is only about adjusting the documentation to target the

Logging in SparkExtensions

Hi, I have created a fat / shaded library jar to use in Spark via SparkExtensions. The usage is done through setting spark.sql.extensions conf to my class that extends `SparkSessionExtensionsProvider` within my jar. The purpose of this extension jar is to inject my custom UDFs functions (see:

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

Thanks for creating the RC1, Xinrong! Besides the blockers mentioned by Tom, let's include the following bug fix in Spark 3.4.0 as well: [SPARK-42406][SQL] Fix check for missing required fields of to_protobuf

Re: [DISCUSS] Show Python code examples first in Spark documentation

If this is not just flip flopping the document pages and involves other changes, then a proper impact analysis needs to be done to assess the efforts involved. Personally I don't think it really matters. HTH view my Linkedin profile