RE: why the pyspark RDD API is so slow?

2022-01-30 Thread Theodore J Griesenbrock
Any particular code sample you can suggest to review on your tips? > On Jan 30, 2022, at 06:16, Sebastian Piu wrote: > >  > This Message Is From an External Sender > This message came from outside your organization. > It's because all data needs to be pickled back and forth between java and a

RE: Is user@spark indexed by google?

2022-01-21 Thread Theodore J Griesenbrock
Try searching here:   https://lists.apache.org/list.html?user@spark.apache.org   -T.J.     T.J. Griesenbrock Technical Release Manager Watson Health He/Him/His   +1 (602) 377-7673 (Text only)t...@ibm.com  IBM     - Original message -From: "Mich Talebzadeh" To:Cc: "user @spark" Subject:

Re: questions on these functions

2022-01-21 Thread Theodore J Griesenbrock
I discovered several instances of discussion on leftFold and rightFold in a variety of forums, but I can not find anything related to RDD in the official documentation:   https://spark.apache.org/docs/latest/api/scala/org/apache/spark/rdd/RDD.html   It appears to be non-related to Spark, and

RE: Does Spark 3.1.2/3.2 support log4j 2.17.1+, and how? your target release day for Spark3.3?

2022-01-19 Thread Theodore J Griesenbrock
can do to ensure we stay up to date with the news.   Thanks!   -T.J.     T.J. Griesenbrock Technical Release Manager Watson Health He/Him/His   +1 (602) 377-7673 (Text only)t...@ibm.com  IBM     - Original message -From: "Sean Owen" To: "Juan Liu" Cc: "Theodor