Extracting Input and Output Partitions in Spark

2024-01-30 Thread Aditya Sohoni
Hello Spark Devs! We are from Uber's Spark team. Our ETL jobs use Spark to read and write from Hive datasets stored in HDFS. The freshness of the partition written to depends on the freshness of the data in the input partition(s). We monitor this freshness score, so that partitions in our

Spark 3.5.1

2024-01-30 Thread Santosh Pingale
Hey there Spark 3.5 branch has accumulated 199 commits with quite a few bug fixes related to correctness. Are there any plans for releasing 3.5.1? Kind regards Santosh

Re: [QUESTION] Legal dependency with Oracle JDBC driver

2024-01-30 Thread Mich Talebzadeh
Hi Alex, Well, that is just Justin's opinion vis-à-vis his matter. It is different from mine. Bottom line, you can always refer to Oracle or a copyright expert on this matter and see what they suggest. HTH Mich Talebzadeh, Dad | Technologist | Solutions Architect | Engineer London United Kingdom