Hi, Just wanna open this up for discussion as it seems we somehow skipped this point. Basically, by now we pretty much have the new datasets APIs in place in the Python SDK and in implementing frameworks. (amaterasu-pyspark, amaterasu-pandas, amaterasu-python) The only question left is regarding the way we get the datasets definitions. Currently, we still look up the datasets definitions in the maki file, under the action's exports. Do we intend to keep it that way? I assume not as I think that every action needs access to all defined datasets. In that case, how will the user submit datasets configuration? Is it another file next to the maki.yaml? Is it a file that resides in the environment, e.g. next to the env.yaml? Is it not even a file on its own but a part of the env.yaml? Ideas, anyone?
Let's discuss this please! Cheers, Nadav
