Hey everyone, I opened a draft PR for Amaterasu 45 (Python SDK, pyspark runtime, python runtime). Do mind that this is still a WIP. I would like you to review this PR from time to time as it evolves. The sooner I get inputs, the sooner this feature will be out.
PR: https://github.com/apache/incubator-amaterasu/pull/44 Yaniv, Eyal, Arun. Please let's now define the final MVP of this feature so I can know which reviews to defer and which to attend to before this PR is closed and merged. Arun, this is most important, if I ended up implementing a design that is vastly different than the scala runtime design, we should talk ASAP. I would like the community to give any feedback. Did I miss something? Did I make some stupid mistake? Does the design look awkward? etc. Also a note on the python runtime. It is currently February 2019. As of January 1st 2020 Python2 is EOL. Many third party Python packages already dropped support entirely for Python2. Because of that, the entire SDK and runtimes are implemented in Python >=3.4 (Python <= 3.3.x is EOL) If you think this is problematic, please raise a flag. Cheers, Nadav
