Re: Spark 2.4.x and 3.x datasourcev2 api documentation & references

2020-03-18 Thread MadDoxX
Thanks for the link ! i was able to code my first datasources in microbatch and continuous by using the v2 api... i was wondering if there are some equivalent to SparkListener for microbatching & continuous stream (where one could use the spark.extra.listener) ~ monitoring :) -- Sent from: htt

Spark 2.4.x and 3.x datasourcev2 api documentation & references

2020-03-15 Thread MadDoxX
Hi, i have been following closely the new proposed api, and seems until now some lack of documentation... specially for people trying to implement them in JAVA. Anyway to give out some reference as starting point for java developpers as tests classes ? (anyway new api looks promising) ! --