Dear TD, Matei, Michael, Reynold, I hope all of you and your loved ones are staying safe and doing well.
as a member of the community the direction from the SPARK mentors is getting to be a bit confusing for me and I was wondering if I can seek your help. We have to make long term decisions which is aligned with the open source SPARK compatibility and directions and it will be wonderful to know what is the most dependable route to get data from SPARK to tensorflow, is it: 1. petastorm 2. horovod 3. tensorflowonspark 4. spark_tensorflow_distributor or something else. Any comments from you will be super useful. If I am not wrong, seamless integration between SPARK to tensorflow/ pytorch was one of the most exciting visions of SPARK 3.x While using SPARK ML has its own favourite space, I think that tensorflow and pytorch will see a lot of focused development as well. Regards, Gourav Sengupta
