This is an automated email from the ASF dual-hosted git repository.
zkaoudi pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-wayang.git
from 1788e1b3e Merge pull request #558 from mspruc/main
add 8d9b931d6 Add parquet operators to the Python API
add 2a201bb55 Update .gitignore
add 06e4a2c85 Update core.py
add ae9d7e1f6 Expose Parquet operators to the Python API
add eaefe68fe Add projection to the serializer
add 291a0297e Merge remote-tracking branch 'origin/main' into
python-implementation
add d5e99a4f9 Merge branch 'apache:main' into python-implementation
add 789c886c6 update Record to string methods
add 2e2279713 add parquet source operator to Python API
add b30258463 remove print statements
add 605d27ed7 remove distinct operator from commit
add 9c8627710 remove serialization of iterator for commit
add 84d8bbd0c revert properties
add 976c709f2 update parquet input from drawflow
add 54706911b remove distinct operator for commit
add 905087f23 remove type deserialization for commit
add 846848228 remove column names
add 00e9ad877 update comment
new 9bad982d7 Merge pull request #559 from aurb9/python-implementation
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
python/.gitignore | 2 +-
python/src/pywy/{ => basic/data}/__init__.py | 0
python/src/pywy/basic/data/record.py | 67 +++++++++++++++++++
python/src/pywy/core/core.py | 11 +--
python/src/pywy/core/serializer.py | 32 +++++----
python/src/pywy/dataquanta.py | 78 ++++++++++++++--------
python/src/pywy/execution/util.py | 1 -
python/src/pywy/execution/worker.py | 17 +++--
python/src/pywy/graph/graph.py | 22 +++---
python/src/pywy/operators/__init__.py | 41 +++++++-----
python/src/pywy/operators/base.py | 23 ++++---
python/src/pywy/operators/binary.py | 47 +++++++------
python/src/pywy/operators/sink.py | 4 +-
python/src/pywy/operators/source.py | 27 +++++++-
python/src/pywy/operators/unary.py | 54 ++++++---------
python/src/pywy/tests/filter_test.py | 4 +-
python/src/pywy/tests/test.py | 3 +-
python/src/pywy/types.py | 34 +++++-----
.../wayang-api-json/src/main/scala/Main.scala | 4 --
.../src/main/scala/builder/JsonPlanBuilder.scala | 28 ++++----
.../OperatorFromDrawflowConverter.scala | 15 ++---
.../scala/operatorfromjson/ContextFromJson.scala | 2 +-
.../scala/operatorfromjson/OperatorFromJson.scala | 35 +++++-----
.../main/scala/operatorfromjson/PlanFromJson.scala | 8 ---
.../binary/IntersectOperatorFromJson.scala | 1 -
.../binary/JoinOperatorFromJson.scala | 3 +-
.../binary/UnionOperatorFromJson.scala | 2 -
...utFromJson.scala => ParquetInputFromJson.scala} | 10 +--
.../unary/DistinctOperatorFromJson.scala | 1 -
.../unary/FilterOperatorFromJson.scala | 3 +-
...romJson.scala => GroupByOperatorFromJson.scala} | 6 +-
.../unary/SortOperatorFromJson.scala | 3 +-
.../function/WrappedTransformationDescriptor.java | 1 -
.../wayang-api-python-defaults.properties | 1 -
.../org/apache/wayang/api/JavaPlanBuilder.scala | 3 +-
.../scala/org/apache/wayang/api/PlanBuilder.scala | 2 +-
.../org/apache/wayang/api/util/NDimArray.scala | 3 +
.../java/org/apache/wayang/basic/data/Record.java | 4 +-
38 files changed, 346 insertions(+), 256 deletions(-)
copy python/src/pywy/{ => basic/data}/__init__.py (100%)
create mode 100644 python/src/pywy/basic/data/record.py
copy
wayang-api/wayang-api-json/src/main/scala/operatorfromjson/input/{TextFileInputFromJson.scala
=> ParquetInputFromJson.scala} (85%)
rename
wayang-api/wayang-api-json/src/main/scala/operatorfromjson/unary/{GroupByOpeartorFromJson.scala
=> GroupByOperatorFromJson.scala} (90%)
