Protocol Meeting 20210125 - “Wayang Roadmap” 1. Clarify Aim / Purpose
Show the roadmap idea to the main developers and discuss the possible paths to complete the roadmap. 2. Review Agenda The agenda for this meeting is looking about the three main point that the work on wayang it will follow: a) Wayang API a. SQL API b. EMMA API c. Python API b) Wayang Core a. Unifications of logs libraries b. Update the libraries c. Increase the coverage of the Unit Test c) Wayang Issues a. Add more examples b. Create a better documentation c. Simplification on the installation and usage of Wayang 3. Work through Agenda The main work made about the agendas was as follows. a) Wayang API *a. SQL API:* the SQL API currently is starting to be developed using Apache Calcite, but because it has two possible ways to be implemented is needed to prepare some pros and cons of the two possible ways to be implemented *b. EMMA API:* The Emma is a DSL language that has the advantage of making some optimization in terms of the nested Parallelism that can improve the performance of Wayang at the execution time, helping the final user at the moment of implementing a complex task. *c. Python API:* the python API has a goal to create an API that can help the usage of different platforms with python even if that platform doesn’t have python API. However, that makes some challenges as to how the design of the code needs to work. b) Wayang Core *a. Unification of logs libraries:* Because in the past the code has several contributors and it was made to do research with it, the logs libraries does not have some protocol, because of that many times is a little bit complex to enable debugging the code. The main goal is to search for the best library for our use case and change the libraries for that one and make the protocol to use the logs inside of the code of wayang. *b. Update libraries: *Many of the libraries that are working with wayang need to be updated because they have a new version and others are deprecated then it is necessary to validate the version and the stability of them. *c. Increase the coverage of the Unit Test:* The current coverage of the unit test is low, and because the code it will be manipulated by a community is mandatory to have a huge coverage test to avoid the new feature breaking the current code implemented. c) Wayang Issues *a. Add more examples: *One problem that Wayang has is the small number of examples and use cases that it has, then if we want to have more users and contributors, we need to add a considerable number of examples. *b. Create a better documentation: *tie to the same reason as the previous point, create a better documentation is mandatory, the new documentation needs to be created in base an example with clear propours in mind. *c. Simplification on the installation and usage of Wayang:* Wayang currently have many platforms but the most used are Spark, Flink and SQL Connections, and it's already complex in some case the installation and the configuration of that number of platforms to be enable to work with Wayang, the idea is to create some “script” that help the installation and deploy of wayang. 4. Review meeting record The current idea is creating 4 teams that it will take care of the 1) SQL API 2) API EMMA 3) Python API 4) Wayang Core and Issue That distribution comes from the idea that having more API available will help in getting more users rather than only java and spark developers. However, in the Wayang core and Issues team, it will be helping another team in modifications that are needed on the core to make them feature works. Images, slides and other comments you can found it on http://wayang.apache.org/meetings/20210125/
