This is an automated email from the ASF dual-hosted git repository.
kamir pushed a change to branch kamir-patch-1
in repository https://gitbox.apache.org/repos/asf/incubator-wayang.git
from 9526794c clean patch
add 830e2eca Wayang 8 (#89)
add 8f7c5beb Add WAYANG-8 with python encapsulation API to fork
add fa5ebab5 Rebase from main
add 94c72826 Insert placeholder UDF into WrappedPythonFunction
add b01e8160 Placeholder cloudpickle lambda inserted
add ee94ed3a Add test.py to debug run the python API
add 3ea4c57e Restructure python modules
add 6f8ce7e9 Modify python code from second pywayang implementation to run
add a5a836fc Fix operator types for JVM so that wayang-api-rest can
interpret them
add 588251cd Integrate with python library and add FlatmapOperator for JVMs
add 59e3e359 Replace PythonUDF in PythonAPITest with placeholder
add 78048acd Add call to action to PYWAYANG.md after strategizing
add 47a8e302 Correct rights on python examples
add 5797512f Add PYWAYANG.md comments
add 7f28e719 Update PYWAYANG.md with newest call-to-action
add c1cf1216 Rebase from main
add ca67db57 Example json_test added to call REST API and execute test
plans
add 22ccc9b0 Add ZIO-http as an alternative to http4s that has cats
conflicts
add 3f2d4a8c Add drawflow and json routes
add c43eff3f Enable python origin for most unary common operators with UDFs
add ae9591f3 Fix serialization failing in spark and execute from base64
pickled python UDFs
add a549ed69 Add first iteration of PywyPlan to json execution
add c77d5747 Add operator data serialization to successfully execute
add 3f7d9c99 Add wordcount python API example
add c9ba4138 Allow reduce_by_key to take key_udf as well
add 8905cf42 Tidy up legacy modules from wayang-8 branch
add 14b95849 Modify python api README.md to hold instructions
add 8550864d Add MapPartitions pipeline wrapping in pywy
add ad9cb683 Fix pipeline wrapping
add 57af3a1d Add wrapped descriptors for python UDFs
add cd01b2ac Finish usage of WrappedTransformationDescriptor
add e8d01ff2 Ignore python build files
add 7ee2620e Remove the placeholder tests
add 164226e1 Remove dotfiles
add a4b4dab7 Remove more dotfiles
add 87058f31 Remove python dist files
add bfdf374b Remove maven-wrapper.jar from changes
add fa5ee03d Remove wayang-api-json/pom conflicts
add e8919133 Merge pull request #425 from apache/kamir-patch-1
add e8485095 Minor changes for compilation with raw mvn
add e7e91b21 Merge pull request #426 from Baunsgaard/MVNBuild
add 830f86ef Add first draft of ml-in-wayang.md
add 65987c25 add ASF license header
add 93f2d839 Merge pull request #398 from juripetersen/ml-model-guide
add 2192b8d7 removed github actions
add be2572d9 removed github actions
add b884e769 removed ASF license
add ef213dd8 Update README.md
add a569d116 Merge pull request #2 from databloom-ai/2pk03-patch-1
add 70afca9a Update README.md
add d8c486b3 Merge pull request #3 from databloom-ai/2pk03-patch-1
add 34b93e82 Update README.md
add 99dd0ed7 Merge pull request #4 from databloom-ai/2pk03-patch-1
add 5834072e [wayang-platforms] Added sql to uncached rdd channel
conversion.
add 93f466ca [wayang-platforms] Small fix for previous commit.
add 301c2ed4 [wayang-platforms] Small tweak for SqlToRddOperator.
add d1f81546 [wayang-api-scala-java] Added initial implementation of
MultiContextPlanBuilder.scala
add 079831c4 [wayang-api-scala-java] MultiContextPlanBuilder: Added
withUdfJars method
add c1cf1871 [wayang-api-scala-java] MultiContextPlanBuilder: A sink is
now bound to a context. Also added a thread per context.
add eca74468 [wayang-api-scala-java] Working on plan serialization
add 87383fd2 [wayang-api-scala-java] Still working on plan serialization
add 991cba1c [wayang-api-scala-java] First working version of plan
serialization
add 88e6ff2c [wayang-api-scala-java] [multi-context-plan-builder] Working
on testing and benchmarks
add bedbe4ef [wayang-api-scala-java] [multi-context-plan-builder] More
operators can be serialized & more tests
add 18153f91 [wayang-api-scala-java] [multi-context-plan-builder] Added a
word count benchmark
add 3996df31 [wayang-api-scala-java] [multi-context-plan-builder] Some
tweaks for running on gcp
add d0242ef6 [wayang-api-scala-java] [multi-context-plan-builder] Added
binary and loop operators & tests
add 7c0ba491 [multi-context-plan-builder] Tweaks for loop and sample
operator serialization
add ba22ec47 [wayang-benchmark] [multi-context-plan-builder] Tweaked
WordCount and added TPCHQuery1
add f31a5535 [wayang-benchmark] [multi-context-plan-builder] TPCHQuery1:
Tweak
add c06afcd1 [multi-context-plan-builder] Added target platform
serialization
add bd663133 [multi-context-plan-builder] Added multiple sources, working
on target platforms
add 3a1ce062 [multi-context-plan-builder] Fixed target platforms
serialization
add 23552888 [multi-context-plan-builder] Refactored benchmarks
add 06d9ca44 [multi-context-plan-builder] Tweaks and cleanup
add 5812f83c [multi-context-plan-builder] Fixed source and output of join
and coGroup test
add 8dcbc92a [multi-context-plan-builder] Cleanup on SerializationUtils
add 02a1db9f [multi-context-plan-builder] Serialization/deserialization
fixes for table sources and connecting loop operators to other operators
add ff046559 [multi-context-plan-builder] Added merging of contexts
add b530ff65 [multi-context-plan-builder] Tweaks in mergeUnion
add 47511ab0 [multi-context-plan-builder] Working on async executions and
async combinations
add a76cafc7 [multi-context-plan-builder] First working example of async
execution
add 98487dee [multi-context-plan-builder] Fixed some sample operator tests
due an earlier tweak for making it serializable
add 578f2c47 [multi-context-plan-builder] Fixed some sample operator tests
due an earlier tweak for making it serializable vol.2
add c5d84eeb [multi-context-plan-builder] [wayang-benchmark] Removed
unnecessary planBuilder instances in parallel wayang job in
multicontext.apps.wordcount.Test
add 35a85635 Added missing apache licence headers and tweaks.
add b28bd45e [multi-context-plan-builder] Working on refactoring
AsyncDataQuanta
add dc074add [multi-context-plan-builder] Added additional APIs for the
asynchronous wayang jobs - still haven't decided on one
add 7e10e975 [multi-context-plan-builder] [asynchronous-jobs] Refactors
and documentation
add 33e337ba [multi-context-plan-builder] [asynchronous-jobs] Refactored
multicontext to use runAsync api. Moved main from MultiContextDataQuanta to
async. Various others refactors.
add 64e6e42b [multi-context-plan-builder] [asynchronous-jobs] Working on
finalizing refactors.
add 875f3a86 [multi-context-plan-builder] [asynchronous-jobs] Finalized
refactors. MultiContextDataQuanta api now runs on top of the async api.
add 425b4dd6 [multi-context-plan-builder] [asynchronous-jobs] Minor tweaks
in async benchmark mains
add d6fa9d8d [multi-context-plan-builder] OperatorSerializationTests:
Deactivated some logs
add 1be18f4b [multi-context-plan-builder] Added serialization support for
load estimators and reorganized mixins for serialization
add ebcfef2b [multi-context-plan-builder] Added serialization support for
cardinality estimators
add 35133b5c [multi-context-plan-builder] Parent process now passes its
own classpath. Async and multiContextPlanBuilder tests are now fixed.
add 724b9204 [multi-context-plan-builder] Tweaks
add c743bbc0 [multi-context-plan-builder] Added documentation (readme.md)
for async and multicontext operations. Also, tweaked the corresponding
benchmarks.
add 29c60770 [multi-context-plan-builder] [wayang-api-scala-java]
README.md tweaks.
add 0c9958ea Merged wayang 0.7.1
add b50656c3 Added readme on how to merge changes from the public wayang
repo
add 38563acf Added licenses
add f27eb6c0 [wayang-api-scala-java] Removed some logs in tests
add 49421909 Merged multicontext
add f4b4a5a5 Merged multicontext tweak
add 070b0d5e Merged multicontext cleanup
add 0d0fae91 Renamed BlossomContext to MultiContext
add b67b404e Merge pull request #429 from damik3/main
add a1081202 Bump org.hsqldb:hsqldb in
/wayang-platforms/wayang-jdbc-template
add b5532bf9 Merge pull request #435 from
apache/dependabot/maven/wayang-platforms/wayang-jdbc-template/org.hsqldb-hsqldb-2.7.1
add 4ba5435d Merge branch 'main' into wayang-python-api
add a834fb99 Fix scala-java dependencies
add 03d61015 Fix license headers
add 647834af Add placeholder for tensorflow scala operators
add c25c6561 Merge pull request #436 from juripetersen/wayang-python-api
add 3af0d5ea Bump com.google.protobuf:protobuf-java
add e6cabb03 Merge pull request #445 from
apache/dependabot/maven/wayang-api/wayang-api-scala-java/com.google.protobuf-protobuf-java-3.16.3
add 81a88cc3 Bump com.google.protobuf:protobuf-java in
/wayang-api/wayang-api-python
add 224714b9 Merge pull request #446 from
apache/dependabot/maven/wayang-api/wayang-api-python/com.google.protobuf-protobuf-java-3.16.3
add 2e24f78e Bump com.google.protobuf:protobuf-java
add 97bed9f8 Merge pull request #451 from
apache/dependabot/maven/wayang-commons/wayang-serializable/com.google.protobuf-protobuf-java-3.16.3
add e3cb017f Rebase from main
add 14755df1 Add GenericTco inputs for all operators
add 81d777f6 Initiate infering types on operators
add 4997dc42 Use optional type inference with default parameters on
operators
add 3499b756 Add model and op infrastructure in python-api
add 9b766d55 Add operators to dataquanta.py
add c523704b Outline json format for predict and DlTraining
add fae64568 Add DLModel and optimizer serialization
add 102753db Parse plan from json with hardcoded model details
add 7946b8d9 Add mappings for Ops parameters
add 5c3ebb0c Remove dotfiles
add 629449d8 Fix license header in dataquanta.py
add 1cb3a049 Try constrained typing that is non functional yet
add c67075e2 Add NDimArray to python types to encode types sent to java
add 5f87720c Add NDimArray typing that to operatorsFromJson
add c0c2cdb4 Add hacky way to use stream of list in NdArray
add a7eb3baa Add NdimArray typing to fit Tensorflow implementation of
DLTraining and Predict operators
add 10e27e13 Add sort operator
add 6bbdcd48 Change action to install locally, then use local build files
to run tests
add 3526e7a0 Add configuration to python API for config ergonomics
add 3fa3c389 Merge pull request #454 from juripetersen/wayang-python-api
add 01bc64db fix: Updated to the latest apache parent pom and removed
duplicate snapshot repo definitions in the root pom (they are re-defined in the
apache parent)
add 7e952ff6 Merge pull request #455 from apache/fix/release-issues
add d5fbe209 added changed files for new feature - read from textfile via
URL
add 68ac072b removed comments
add 30850169 added missing file and removed comments
add 54f6f47f Merge pull request #459 from kamir/mk-feature-1
add 24b660b4 Update README.md to test commit permissions on repo
add 0d4a319f Merge pull request #427 from apache/kamir-patch-2
add e81e9409 Set logLevel on WayangContext to allow doing it without conf
file
add 09504efa Merge pull request #453 from
juripetersen/configurable-log-level
add 6c33ebe0 Update README.md
add 88dce06c Merge pull request #462 from apache/2pk03-patch-2
add 24487667 Merge branch 'main' into kamir-patch-1
No new revisions were added by this update.
Summary of changes:
.github/workflows/backend.yml | 4 +-
.gitignore | 6 +-
README.md | 4 +-
build/pyplangenerator.sh | 4 +-
conf/spark/default.properties | 3 +-
guides/ml-in-Wayang.md | 248 +
pom.xml | 44 +-
python/.gitignore | 278 +-
python/README.md | 75 +
python/{ => old_code}/.gitignore | 0
python/{ => old_code}/.rat-excludes | 0
python/{ => old_code}/LICENSE | 0
python/{ => old_code}/README.md | 0
python/{ => old_code}/bin/benchmark.sh | 0
python/{ => old_code}/bin/test.sh | 0
python/old_code/{ => old_code}/__init__.py | 0
.../{ => old_code}/config/config_reader.py | 0
.../{ => old_code}/config/pywayang_config.ini | 0
.../old_code/{ => old_code}/old_graph/__init__.py | 0
python/old_code/{ => old_code}/old_graph/graph.py | 0
python/old_code/{ => old_code}/old_graph/node.py | 0
.../old_code/{ => old_code}/old_graph/traversal.py | 0
.../old_code/{ => old_code}/old_graph/visitant.py | 0
.../{ => old_code}/orchestrator/__init__.py | 0
.../{ => old_code}/orchestrator/dataquanta.py | 0
.../{ => old_code}/orchestrator/execdirectly.py | 0
.../old_code/{ => old_code}/orchestrator/main.py | 0
.../{ => old_code}/orchestrator/operator.py | 0
.../old_code/{ => old_code}/orchestrator/plan.py | 0
.../old_code/{ => old_code}/protobuf/__init__.py | 0
.../old_code/{ => old_code}/protobuf/planwriter.py | 0
python/old_code/{ => old_code}/test.py | 0
python/old_code/{ => old_code}/tests/__init__.py | 0
.../old_code/{ => old_code}/tests/demo_testing.py | 0
.../{ => old_code}/tests/full_java_test.py | 0
.../{ => old_code}/tests/full_spark_test.py | 0
python/{pywayang.toml => old_code/pyproject.toml} | 2 +-
.../pywayang/pyproject.toml} | 2 +-
python/{ => old_code/pywayang}/setup.cfg | 6 +-
.../old_code/pywayang/src/pywy/__init__.py | 2 +-
.../old_code/pywayang/src/pywy/config/__init__.py | 4 +
.../pywayang/src/pywy/config/config_reader.py | 55 +
.../pywayang/src/pywy/config/pywayang_config.ini | 26 +-
.../old_code/pywayang/src/pywy/graph/__init__.py | 7 +-
.../src/pywy/graph}/graph.py | 26 +-
.../{old_graph => pywayang/src/pywy/graph}/node.py | 24 +-
.../src/pywy/graph}/traversal.py | 26 +-
.../src/pywy/graph}/visitant.py | 24 +-
.../pywayang/src/pywy/orchestrator/__init__.py | 7 +-
.../src/pywy}/orchestrator/dataquanta.py | 36 +-
.../src/pywy}/orchestrator/execdirectly.py | 28 +-
.../{ => pywayang/src/pywy}/orchestrator/main.py | 29 +-
.../src/pywy}/orchestrator/operator.py | 38 +-
.../{ => pywayang/src/pywy}/orchestrator/plan.py | 25 +-
.../pywayang/src/pywy/protobuf/__init__.py | 1 +
.../pywayang/src/pywy/protobuf/old_planwriter.py | 308 +
.../{ => pywayang/src/pywy}/protobuf/planwriter.py | 27 +-
.../pywayang/src/pywy/test/demo_testing.py | 13 +
.../pywayang/src/pywy/test/full_java_test.py | 68 +
.../src/pywy/test}/full_spark_test.py | 34 +-
python/{ => old_code}/resources/test.input | 0
python/{ => old_code}/resources/text.input | 0
python/{ => old_code}/setup.cfg | 10 +-
.../jvm => old_code/src/pywy}/__init__.py | 0
python/{ => old_code}/src/pywy/config.py | 0
python/{ => old_code}/src/pywy/core/__init__.py | 0
python/{ => old_code}/src/pywy/core/channel.py | 0
python/{ => old_code}/src/pywy/core/core.py | 0
python/{ => old_code}/src/pywy/core/executor.py | 0
python/{ => old_code}/src/pywy/core/mapping.py | 0
python/{ => old_code}/src/pywy/core/platform.py | 0
python/{ => old_code}/src/pywy/dataquanta.py | 0
python/{ => old_code}/src/pywy/exception.py | 0
.../src/pywy/graph}/__init__.py | 0
python/{ => old_code}/src/pywy/graph/graph.py | 0
python/{ => old_code}/src/pywy/graph/types.py | 0
.../{ => old_code}/src/pywy/operators/__init__.py | 0
python/{ => old_code}/src/pywy/operators/base.py | 0
python/{ => old_code}/src/pywy/operators/sink.py | 0
python/{ => old_code}/src/pywy/operators/source.py | 0
python/{ => old_code}/src/pywy/operators/unary.py | 0
.../src/pywy/platforms}/__init__.py | 0
.../src/pywy/platforms/commons/__init__.py | 0
.../src/pywy/platforms/commons/channels.py | 0
.../src/pywy/platforms/jvm}/__init__.py | 0
.../src/pywy/platforms/jvm/channels.py | 0
.../src/pywy/platforms/jvm/context.py | 0
.../src/pywy/platforms/jvm/execution.py | 0
.../{ => old_code}/src/pywy/platforms/jvm/graph.py | 0
.../src/pywy/platforms/jvm/mappings.py | 1 +
.../src/pywy/platforms/jvm/operator/__init__.py | 2 +
.../jvm/operator/jvm_execution_operator.py | 0
.../platforms/jvm/operator/jvm_sink_textfile.py | 0
.../platforms/jvm/operator/jvm_source_textfile.py | 0
.../platforms/jvm/operator/jvm_unary_filter.py | 0
.../platforms/jvm/operator/jvm_unary_flatmap.py} | 15 +-
.../src/pywy/platforms/jvm/platform.py | 0
.../src/pywy/platforms/jvm/plugin.py | 0
.../pywy/platforms/jvm/serializable}/__init__.py | 0
.../platforms/jvm/serializable/plan_writter.py | 0
.../jvm/serializable/wayang_jvm_operator.py | 6 +-
.../src/pywy/platforms/jvm/worker.py | 5 +-
.../src/pywy/platforms/python}/__init__.py | 0
.../src/pywy/platforms/python/channels.py | 0
.../src/pywy/platforms/python/execution.py | 0
.../src/pywy/platforms/python/mappings.py | 0
.../src/pywy/platforms/python/operator/__init__.py | 0
.../python/operator/py_execution_operator.py | 0
.../platforms/python/operator/py_sink_textfile.py | 0
.../python/operator/py_source_textfile.py | 0
.../platforms/python/operator/py_unary_filter.py | 0
.../platforms/python/operator/py_unary_flatmap.py | 0
.../pywy/platforms/python/operator/py_unary_map.py | 0
.../src/pywy/platforms/python/platform.py | 0
.../src/pywy/platforms/python/plugin.py | 0
python/{ => old_code}/src/pywy/plugins.py | 0
.../src/pywy/tests}/__init__.py | 0
.../src/pywy/tests/benchmark/__init__.py | 0
.../pywy/tests/benchmark/python_benchmark_test.py | 1 +
.../src/pywy/tests/integration/__init__.py | 0
.../pywy/tests/integration/jvm_platform_test.py | 0
.../pywy/tests/integration/python_platform_test.py | 0
.../src/pywy/tests/unit}/__init__.py | 0
.../src/pywy/tests/unit/core}/__init__.py | 0
.../src/pywy/tests/unit/core/channel_test.py | 0
.../src/pywy/tests/unit/core/executor_test.py | 0
.../src/pywy/tests/unit/core/mapping_test.py | 0
.../src/pywy/tests/unit/core/plan_test.py | 0
.../src/pywy/tests/unit/core/platform_test.py | 0
.../src/pywy/tests/unit/core/plugin_test.py | 0
.../src/pywy/tests/unit/core/translator_test.py | 0
.../src/pywy/tests/unit/dataquanta/__init__.py | 0
.../src/pywy/tests/unit/dataquanta/context_test.py | 0
.../pywy/tests/unit/dataquanta/dataquanta_test.py | 0
.../src/pywy/tests/unit/graph}/__init__.py | 0
.../src/pywy/tests/unit/graph/graph_test.py | 0
.../src/pywy/tests/unit/graph/types_test.py | 0
.../src/pywy/tests/unit/operators}/__init__.py | 0
.../src/pywy/tests/unit/operators/base_test.py | 0
.../src/pywy/tests/unit/operators/sink_test.py | 0
.../src/pywy/tests/unit/operators/source_test.py | 0
.../src/pywy/tests/unit/operators/unary_test.py | 0
.../src/pywy/tests/unit/platforms}/__init__.py | 0
.../pywy/tests/unit/platforms/python}/__init__.py | 0
.../tests/unit/platforms/python/channel_test.py | 0
.../tests/unit/platforms/python/execution_test.py | 0
.../tests/unit/platforms/python/mappings_test.py | 0
.../unit/platforms/python/operator}/__init__.py | 0
.../python/operator/py_execution_operator_test.py | 0
.../python/operator/py_sink_textfile_test.py | 0
.../python/operator/py_source_textfile_test.py | 0
.../python/operator/py_unary_filter_test.py | 0
.../tests/unit/platforms/python/platform_test.py | 0
.../tests/unit/platforms/python/plugin_test.py | 0
.../src/pywy/tests/unit/types_test.py | 0
.../src/pywy/translate}/__init__.py | 0
python/{ => old_code}/src/pywy/types.py | 0
python/{pywayang.toml => pyproject.toml} | 2 +-
python/setup.cfg | 12 +-
python/src/.gitignore | 1 +
python/src/pywy/{ => basic}/__init__.py | 0
python/src/pywy/{ => basic/model}/__init__.py | 0
.../python/platform.py => basic/model/models.py} | 13 +-
python/src/pywy/basic/model/ops.py | 235 +
python/src/pywy/basic/model/optimizer.py | 63 +
python/src/pywy/basic/model/option.py | 55 +
python/src/pywy/core/__init__.py | 10 +-
.../pywy/core/__pycache__/__init__.cpython-38.pyc | Bin 0 -> 279 bytes
.../src/pywy/core/__pycache__/core.cpython-38.pyc | Bin 0 -> 4921 bytes
.../pywy/core/__pycache__/platform.cpython-38.pyc | Bin 0 -> 932 bytes
python/src/pywy/core/core.py | 142 +-
.../python/platform.py => core/functions.py} | 17 +-
python/src/pywy/core/serializer.py | 114 +
python/src/pywy/dataquanta.py | 127 +-
python/src/pywy/{ => execution}/__init__.py | 0
.../src/pywy/execution/util.py | 9 +
python/src/pywy/execution/worker.py | 138 +
.../pywy/graph/__pycache__/__init__.cpython-38.pyc | Bin 0 -> 138 bytes
.../pywy/graph/__pycache__/graph.cpython-38.pyc | Bin 0 -> 2897 bytes
.../pywy/graph/__pycache__/types.cpython-38.pyc | Bin 0 -> 2963 bytes
python/src/pywy/operators/__init__.py | 15 +-
.../operators/__pycache__/__init__.cpython-38.pyc | Bin 0 -> 521 bytes
.../pywy/operators/__pycache__/base.cpython-38.pyc | Bin 0 -> 2594 bytes
.../pywy/operators/__pycache__/sink.cpython-38.pyc | Bin 0 -> 1831 bytes
.../operators/__pycache__/source.cpython-38.pyc | Bin 0 -> 1784 bytes
.../operators/__pycache__/unary.cpython-38.pyc | Bin 0 -> 5211 bytes
python/src/pywy/operators/base.py | 48 +-
python/src/pywy/operators/binary.py | 89 +
python/src/pywy/operators/sink.py | 12 +-
python/src/pywy/operators/source.py | 4 +
python/src/pywy/operators/unary.py | 136 +-
.../platforms/__pycache__/__init__.cpython-38.pyc | Bin 0 -> 142 bytes
.../pywy/platforms/__pycache__/java.cpython-38.pyc | Bin 0 -> 612 bytes
.../platforms/__pycache__/spark.cpython-38.pyc | Bin 0 -> 618 bytes
.../pywy/platforms/{python/platform.py => java.py} | 8 +-
.../platforms/{python/platform.py => python.py} | 3 +
.../platforms/{python/platform.py => spark.py} | 7 +-
.../{python/platform.py => tensorflow.py} | 8 +-
python/src/pywy/tests/filter_test.py | 42 +
.../pywy/tests/json_test.py} | 35 +-
python/src/pywy/tests/resources/10e0MB.input | 16872 -------------------
python/src/pywy/tests/test.py | 79 +
python/src/pywy/tests/word_count.py | 72 +
python/src/pywy/types.py | 153 +-
wayang-api/wayang-api-json/pom.xml | 128 +-
.../src/main/resources/context-plan-a.json | 81 +
.../wayang-api-json/src/main/resources/plan-a.json | 113 +-
.../wayang-api-json/src/main/scala/Main.scala | 118 +-
.../src/main/scala/builder/JsonPlanBuilder.scala | 428 +-
.../scala/exception/WayangApiJsonException.scala | 1 -
.../OperatorFromDrawflow.scala | 1 -
.../OperatorFromDrawflowConverter.scala | 59 +-
.../ComposedOperatorFromJson.scala | 1 -
.../scala/operatorfromjson/ContextFromJson.scala | 70 +
.../scala/operatorfromjson/OperatorFromJson.scala | 29 +-
.../main/scala/operatorfromjson/PlanFromJson.scala | 69 +
.../binary/CartesianOperatorFromJson.scala | 12 +-
.../binary/CoGroupOperatorFromJson.scala | 14 +-
.../binary/DLTrainingOperatorFromJson.scala | 95 +
.../binary/IntersectOperatorFromJson.scala | 12 +-
.../binary/JoinOperatorFromJson.scala | 6 +-
...romJson.scala => PredictOperatorFromJson.scala} | 15 +-
.../binary/UnionOperatorFromJson.scala | 12 +-
.../input/InputCollectionFromJson.scala | 16 +-
.../input/JDBCRemoteInputFromJson.scala | 6 +-
.../input/TableInputFromJson.scala | 14 +-
.../input/TextFileInputFromJson.scala | 10 +-
.../loop/DoWhileOperatorFromJson.scala | 6 +-
.../loop/ForeachOperatorFromJson.scala | 6 +-
.../loop/RepeatOperatorFromJson.scala | 14 +-
.../operatorfromjson/other/KMeansFromJson.scala | 6 +-
.../output/TextFileOutputFromJson.scala | 10 +-
.../unary/CountOperatorFromJson.scala | 12 +-
.../unary/DistinctOperatorFromJson.scala | 12 +-
.../unary/FilterOperatorFromJson.scala | 6 +-
.../unary/FlatMapOperatorFromJson.scala | 17 +-
.../unary/GroupByOpeartorFromJson.scala | 14 +-
.../unary/MapOperatorFromJson.scala | 13 +-
.../unary/MapPartitionsOperatorFromJson.scala | 17 +-
.../unary/ReduceByOperatorFromJson.scala | 14 +-
.../unary/ReduceOperatorFromJson.scala | 17 +-
.../unary/SampleOperatorFromJson.scala | 6 +-
.../unary/SortOperatorFromJson.scala | 14 +-
.../parserutil/ParseOperatorsFromDrawflow.scala | 1 -
.../scala/parserutil/ParseOperatorsFromJson.scala | 1 -
...atorsFromJson.scala => ParsePlanFromJson.scala} | 16 +-
.../scala/parserutil/ParseScalaFromString.scala | 122 -
.../{MyTypeName.scala => ParsingErrors.scala} | 21 +-
...MyTypeName.scala => SerializableIterable.scala} | 30 +-
.../{MyTypeName.scala => SerializableLambda.scala} | 31 +-
...{MyTypeName.scala => SerializableLambda2.scala} | 30 +-
.../scala/springboot/SpringBootApplication.scala | 31 -
.../scala/springboot/restapi/WayangApiJson.scala | 82 -
wayang-api/wayang-api-python/pom.xml | 37 +-
.../wayang/api/python/executor/ProcessFeeder.java | 48 +-
.../api/python/executor/ProcessReceiver.java | 6 +-
.../api/python/executor/PythonProcessCaller.java | 59 +-
.../api/python/executor/PythonWorkerManager.java | 32 +-
.../wayang/api/python/executor/ReaderIterator.java | 11 +-
.../python/function/WrappedFlatMapDescriptor.java | 53 +
...er.java => WrappedMapPartitionsDescriptor.java} | 46 +-
.../function/WrappedPredicateDescriptor.java | 58 +
.../{PythonUDF.java => WrappedPythonFunction.java} | 17 +-
.../python/function/WrappedReduceDescriptor.java | 54 +
...r.java => WrappedTransformationDescriptor.java} | 46 +-
.../wayang-api-python-defaults.properties | 5 +-
.../server/spring/decoder/WayangPlanBuilder.java | 239 -
.../server/spring/general/WayangController.java | 304 -
wayang-api/wayang-api-scala-java/README.md | 364 +
wayang-api/wayang-api-scala-java/pom.xml | 37 +-
.../scala/org/apache/wayang/api/DataQuanta.scala | 156 +-
.../org/apache/wayang/api/DataQuantaBuilder.scala | 8 +-
.../org/apache/wayang/api/JavaPlanBuilder.scala | 9 +
.../scala/org/apache/wayang/api/MultiContext.scala | 73 +
.../apache/wayang/api/MultiContextDataQuanta.scala | 179 +
.../wayang/api/MultiContextPlanBuilder.scala | 187 +
.../scala/org/apache/wayang/api/PlanBuilder.scala | 12 +-
.../wayang/api/async/DataQuantaAsyncResult.scala} | 27 +-
.../wayang/api/async/DataQuantaImplicits.scala | 99 +
.../scala/org/apache/wayang/api/async/Main.scala | 85 +
.../wayang/api/async/PlanBuilderImplicits.scala | 49 +
.../org/apache/wayang/api/async/package.scala | 140 +
.../main/scala/org/apache/wayang/api/package.scala | 12 +-
.../api/serialization/SerializationUtils.scala | 203 +
.../wayang/api/serialization/TempFileUtils.scala | 65 +
.../GenericSerializableDeserializer.scala} | 21 +-
.../GenericSerializableSerializer.scala} | 21 +-
.../MultiContextDeserializer.scala | 98 +
.../customserializers/MultiContextSerializer.scala | 63 +
.../customserializers/OperatorDeserializer.scala | 430 +
.../customserializers/PlatformDeserializer.scala | 47 +
.../customserializers/PlatformSerializer.scala} | 18 +-
.../mixins/ConfigurationAndContextMixIns.scala | 122 +
.../api/serialization/mixins/DataTypeMixIns.scala | 68 +
.../serialization/mixins/DescriptorMixIns.scala | 118 +
.../api/serialization/mixins/EstimatorMixIns.scala | 124 +
.../serialization/mixins/IgnoreLoggerMixIn.scala} | 12 +-
.../api/serialization/mixins/OperatorMixIns.scala | 134 +
.../api/serialization/mixins/ProviderMixIns.scala | 85 +
.../api/serialization/mixins/SlotMixIns.scala | 45 +
.../org/apache/wayang/api/util/NDimArray.scala | 52 +
.../serialization/OperatorSerializationTests.scala | 516 +
.../serialization/OtherSerializationTests.scala | 370 +
.../api/serialization/SerializationTestBase.scala | 150 +
wayang-assembly/.gitignore | 1 +
wayang-assembly/.rat-excludes | 2 +
wayang-assembly/dependency-reduced-pom.xml | 180 +
wayang-assembly/pom.xml | 12 +-
wayang-benchmark/pom.xml | 5 +
.../org/apache/wayang/apps/util/Parameters.scala | 3 +
.../org/apache/wayang/async/apps/WordCount.scala | 75 +
.../apache/wayang/multicontext/apps/package.scala | 32 +-
.../wayang/multicontext/apps/tpch/Query1.scala | 131 +
.../multicontext/apps/wordcount/WordCount.scala | 71 +
.../apps/wordcount/WordCountCombineEach.scala | 69 +
.../apps/wordcount/WordCountWithMerge.scala | 79 +
.../wordcount/WordCountWithTargetPlatforms.scala | 69 +
.../basic/operators/PyWayangReduceByOperator.java | 62 +
.../wayang/basic/operators/SampleOperator.java | 17 +-
.../org/apache/wayang/core/api/WayangContext.java | 13 +
.../wayang/core/function/FunctionDescriptor.java | 31 +-
.../cardinality/CardinalityEstimatorManager.java | 2 +-
.../cardinality/DefaultCardinalityEstimator.java | 16 +-
.../core/optimizer/costs/DefaultLoadEstimator.java | 5 +-
.../wayang/core/optimizer/costs/LoadEstimator.java | 3 +-
.../optimizer/costs/LoadProfileEstimators.java | 10 +-
.../costs/NestableLoadProfileEstimator.java | 7 +-
.../core/platform/CrossPlatformExecutor.java | 2 +-
.../apache/wayang/core/util/ReflectionUtils.java | 2 +-
.../DefaultCardinalityEstimatorTest.java | 9 +-
wayang-docs/pom.xml | 6 +-
.../flink/operators/FlinkSampleOperator.java | 3 +-
.../java/operators/JavaRandomSampleOperator.java | 3 +-
.../operators/JavaReservoirSampleOperator.java | 3 +-
.../wayang/java/operators/JavaTextFileSource.java | 74 +-
wayang-platforms/wayang-jdbc-template/pom.xml | 7 +-
.../wayang/jdbc/operators/SqlToRddOperator.java | 5 +-
wayang-platforms/wayang-postgres/pom.xml | 5 +
.../wayang/spark/compiler/FunctionCompiler.java | 2 +
.../compiler/MapPartitionsFunctionAdapter.java | 2 +-
.../operators/SparkBernoulliSampleOperator.java | 3 +-
.../SparkRandomPartitionSampleOperator.java | 3 +-
.../SparkShufflePartitionSampleOperator.java | 3 +-
.../spark/operators/SparkTextFileSource.java | 2 +-
.../wayang/tensorflow/model/TensorflowModel.java | 2 +-
.../operators/TensorflowCollectionSource.java | 9 +-
346 files changed, 8970 insertions(+), 19188 deletions(-)
mode change 100644 => 100755 build/pyplangenerator.sh
create mode 100644 guides/ml-in-Wayang.md
copy python/{ => old_code}/.gitignore (100%)
rename python/{ => old_code}/.rat-excludes (100%)
copy python/{ => old_code}/LICENSE (100%)
copy python/{ => old_code}/README.md (100%)
rename python/{ => old_code}/bin/benchmark.sh (100%)
rename python/{ => old_code}/bin/test.sh (100%)
rename python/old_code/{ => old_code}/__init__.py (100%)
rename python/old_code/{ => old_code}/config/config_reader.py (100%)
copy python/old_code/{ => old_code}/config/pywayang_config.ini (100%)
rename python/old_code/{ => old_code}/old_graph/__init__.py (100%)
copy python/old_code/{ => old_code}/old_graph/graph.py (100%)
copy python/old_code/{ => old_code}/old_graph/node.py (100%)
copy python/old_code/{ => old_code}/old_graph/traversal.py (100%)
copy python/old_code/{ => old_code}/old_graph/visitant.py (100%)
rename python/old_code/{ => old_code}/orchestrator/__init__.py (100%)
copy python/old_code/{ => old_code}/orchestrator/dataquanta.py (100%)
copy python/old_code/{ => old_code}/orchestrator/execdirectly.py (100%)
copy python/old_code/{ => old_code}/orchestrator/main.py (100%)
copy python/old_code/{ => old_code}/orchestrator/operator.py (100%)
copy python/old_code/{ => old_code}/orchestrator/plan.py (100%)
rename python/old_code/{ => old_code}/protobuf/__init__.py (100%)
copy python/old_code/{ => old_code}/protobuf/planwriter.py (100%)
rename python/old_code/{ => old_code}/test.py (100%)
rename python/old_code/{ => old_code}/tests/__init__.py (100%)
rename python/old_code/{ => old_code}/tests/demo_testing.py (100%)
rename python/old_code/{ => old_code}/tests/full_java_test.py (100%)
copy python/old_code/{ => old_code}/tests/full_spark_test.py (100%)
copy python/{pywayang.toml => old_code/pyproject.toml} (95%)
copy python/{pywayang.toml => old_code/pywayang/pyproject.toml} (95%)
copy python/{ => old_code/pywayang}/setup.cfg (96%)
rename wayang-api/wayang-api-json/src/main/resources/application.properties =>
python/old_code/pywayang/src/pywy/__init__.py (93%)
copy wayang-api/wayang-api-json/wayang.properties =>
python/old_code/pywayang/src/pywy/config/__init__.py (85%)
create mode 100644 python/old_code/pywayang/src/pywy/config/config_reader.py
copy build/pyplangenerator.sh =>
python/old_code/pywayang/src/pywy/config/pywayang_config.ini (72%)
rename wayang-api/wayang-api-json/src/main/resources/wayang.properties =>
python/old_code/pywayang/src/pywy/graph/__init__.py (88%)
rename python/old_code/{old_graph => pywayang/src/pywy/graph}/graph.py (68%)
rename python/old_code/{old_graph => pywayang/src/pywy/graph}/node.py (53%)
rename python/old_code/{old_graph => pywayang/src/pywy/graph}/traversal.py
(54%)
rename python/old_code/{old_graph => pywayang/src/pywy/graph}/visitant.py (63%)
copy wayang-api/wayang-api-json/wayang.properties =>
python/old_code/pywayang/src/pywy/orchestrator/__init__.py (87%)
rename python/old_code/{ => pywayang/src/pywy}/orchestrator/dataquanta.py (89%)
rename python/old_code/{ => pywayang/src/pywy}/orchestrator/execdirectly.py
(82%)
rename python/old_code/{ => pywayang/src/pywy}/orchestrator/main.py (83%)
rename python/old_code/{ => pywayang/src/pywy}/orchestrator/operator.py (71%)
rename python/old_code/{ => pywayang/src/pywy}/orchestrator/plan.py (50%)
copy wayang-api/wayang-api-json/wayang.properties =>
python/old_code/pywayang/src/pywy/protobuf/__init__.py (96%)
create mode 100644 python/old_code/pywayang/src/pywy/protobuf/old_planwriter.py
rename python/old_code/{ => pywayang/src/pywy}/protobuf/planwriter.py (92%)
copy wayang-api/wayang-api-json/wayang.properties =>
python/old_code/pywayang/src/pywy/test/demo_testing.py (76%)
create mode 100644 python/old_code/pywayang/src/pywy/test/full_java_test.py
rename python/old_code/{tests => pywayang/src/pywy/test}/full_spark_test.py
(54%)
rename python/{ => old_code}/resources/test.input (100%)
rename python/{ => old_code}/resources/text.input (100%)
copy python/{ => old_code}/setup.cfg (93%)
rename python/{src/pywy/platforms/jvm => old_code/src/pywy}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/config.py (100%)
copy python/{ => old_code}/src/pywy/core/__init__.py (100%)
rename python/{ => old_code}/src/pywy/core/channel.py (100%)
copy python/{ => old_code}/src/pywy/core/core.py (100%)
rename python/{ => old_code}/src/pywy/core/executor.py (100%)
rename python/{ => old_code}/src/pywy/core/mapping.py (100%)
copy python/{ => old_code}/src/pywy/core/platform.py (100%)
copy python/{ => old_code}/src/pywy/dataquanta.py (100%)
copy python/{ => old_code}/src/pywy/exception.py (100%)
rename python/{src/pywy/platforms/jvm/serializable =>
old_code/src/pywy/graph}/__init__.py (100%)
copy python/{ => old_code}/src/pywy/graph/graph.py (100%)
copy python/{ => old_code}/src/pywy/graph/types.py (100%)
copy python/{ => old_code}/src/pywy/operators/__init__.py (100%)
copy python/{ => old_code}/src/pywy/operators/base.py (100%)
copy python/{ => old_code}/src/pywy/operators/sink.py (100%)
copy python/{ => old_code}/src/pywy/operators/source.py (100%)
copy python/{ => old_code}/src/pywy/operators/unary.py (100%)
rename python/{src/pywy/platforms/python =>
old_code/src/pywy/platforms}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/platforms/commons/__init__.py (100%)
rename python/{ => old_code}/src/pywy/platforms/commons/channels.py (100%)
rename python/{src/pywy/tests/unit =>
old_code/src/pywy/platforms/jvm}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/platforms/jvm/channels.py (100%)
rename python/{ => old_code}/src/pywy/platforms/jvm/context.py (100%)
rename python/{ => old_code}/src/pywy/platforms/jvm/execution.py (100%)
rename python/{ => old_code}/src/pywy/platforms/jvm/graph.py (100%)
rename python/{ => old_code}/src/pywy/platforms/jvm/mappings.py (95%)
rename python/{ => old_code}/src/pywy/platforms/jvm/operator/__init__.py (92%)
rename python/{ =>
old_code}/src/pywy/platforms/jvm/operator/jvm_execution_operator.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/jvm/operator/jvm_sink_textfile.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/jvm/operator/jvm_source_textfile.py (100%)
copy python/{ => old_code}/src/pywy/platforms/jvm/operator/jvm_unary_filter.py
(100%)
rename python/{src/pywy/platforms/jvm/operator/jvm_unary_filter.py =>
old_code/src/pywy/platforms/jvm/operator/jvm_unary_flatmap.py} (86%)
rename python/{ => old_code}/src/pywy/platforms/jvm/platform.py (100%)
rename python/{ => old_code}/src/pywy/platforms/jvm/plugin.py (100%)
rename python/{src/pywy/tests/unit/core =>
old_code/src/pywy/platforms/jvm/serializable}/__init__.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/jvm/serializable/plan_writter.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/jvm/serializable/wayang_jvm_operator.py (95%)
rename python/{ => old_code}/src/pywy/platforms/jvm/worker.py (98%)
rename python/{src/pywy/tests/unit/graph =>
old_code/src/pywy/platforms/python}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/platforms/python/channels.py (100%)
rename python/{ => old_code}/src/pywy/platforms/python/execution.py (100%)
rename python/{ => old_code}/src/pywy/platforms/python/mappings.py (100%)
rename python/{ => old_code}/src/pywy/platforms/python/operator/__init__.py
(100%)
rename python/{ =>
old_code}/src/pywy/platforms/python/operator/py_execution_operator.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/python/operator/py_sink_textfile.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/python/operator/py_source_textfile.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/python/operator/py_unary_filter.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/python/operator/py_unary_flatmap.py (100%)
rename python/{ =>
old_code}/src/pywy/platforms/python/operator/py_unary_map.py (100%)
copy python/{ => old_code}/src/pywy/platforms/python/platform.py (100%)
rename python/{ => old_code}/src/pywy/platforms/python/plugin.py (100%)
rename python/{ => old_code}/src/pywy/plugins.py (100%)
rename python/{src/pywy/tests/unit/operators =>
old_code/src/pywy/tests}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/benchmark/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/benchmark/python_benchmark_test.py
(99%)
rename python/{ => old_code}/src/pywy/tests/integration/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/integration/jvm_platform_test.py
(100%)
rename python/{ =>
old_code}/src/pywy/tests/integration/python_platform_test.py (100%)
rename python/{src/pywy/tests/unit/platforms =>
old_code/src/pywy/tests/unit}/__init__.py (100%)
rename python/{src/pywy/tests/unit/platforms/python =>
old_code/src/pywy/tests/unit/core}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/channel_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/executor_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/mapping_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/plan_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/platform_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/plugin_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/core/translator_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/dataquanta/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/dataquanta/context_test.py
(100%)
rename python/{ => old_code}/src/pywy/tests/unit/dataquanta/dataquanta_test.py
(100%)
rename python/{src/pywy/tests/unit/platforms/python/operator =>
old_code/src/pywy/tests/unit/graph}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/graph/graph_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/graph/types_test.py (100%)
rename python/{src/pywy/translate =>
old_code/src/pywy/tests/unit/operators}/__init__.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/operators/base_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/operators/sink_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/operators/source_test.py
(100%)
rename python/{ => old_code}/src/pywy/tests/unit/operators/unary_test.py (100%)
copy python/{src/pywy => old_code/src/pywy/tests/unit/platforms}/__init__.py
(100%)
copy python/{src/pywy =>
old_code/src/pywy/tests/unit/platforms/python}/__init__.py (100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/channel_test.py (100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/execution_test.py (100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/mappings_test.py (100%)
copy python/{src/pywy =>
old_code/src/pywy/tests/unit/platforms/python/operator}/__init__.py (100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/operator/py_execution_operator_test.py
(100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/operator/py_sink_textfile_test.py
(100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/operator/py_source_textfile_test.py
(100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/operator/py_unary_filter_test.py
(100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/platform_test.py (100%)
rename python/{ =>
old_code}/src/pywy/tests/unit/platforms/python/plugin_test.py (100%)
rename python/{ => old_code}/src/pywy/tests/unit/types_test.py (100%)
copy python/{src/pywy => old_code/src/pywy/translate}/__init__.py (100%)
copy python/{ => old_code}/src/pywy/types.py (100%)
rename python/{pywayang.toml => pyproject.toml} (95%)
create mode 100644 python/src/.gitignore
copy python/src/pywy/{ => basic}/__init__.py (100%)
copy python/src/pywy/{ => basic/model}/__init__.py (100%)
copy python/src/pywy/{platforms/python/platform.py => basic/model/models.py}
(81%)
create mode 100644 python/src/pywy/basic/model/ops.py
create mode 100644 python/src/pywy/basic/model/optimizer.py
create mode 100644 python/src/pywy/basic/model/option.py
create mode 100644 python/src/pywy/core/__pycache__/__init__.cpython-38.pyc
create mode 100644 python/src/pywy/core/__pycache__/core.cpython-38.pyc
create mode 100644 python/src/pywy/core/__pycache__/platform.cpython-38.pyc
copy python/src/pywy/{platforms/python/platform.py => core/functions.py} (72%)
create mode 100644 python/src/pywy/core/serializer.py
copy python/src/pywy/{ => execution}/__init__.py (100%)
rename wayang-api/wayang-api-json/wayang.properties =>
python/src/pywy/execution/util.py (81%)
create mode 100755 python/src/pywy/execution/worker.py
create mode 100644 python/src/pywy/graph/__pycache__/__init__.cpython-38.pyc
create mode 100644 python/src/pywy/graph/__pycache__/graph.cpython-38.pyc
create mode 100644 python/src/pywy/graph/__pycache__/types.cpython-38.pyc
create mode 100644
python/src/pywy/operators/__pycache__/__init__.cpython-38.pyc
create mode 100644 python/src/pywy/operators/__pycache__/base.cpython-38.pyc
create mode 100644 python/src/pywy/operators/__pycache__/sink.cpython-38.pyc
create mode 100644 python/src/pywy/operators/__pycache__/source.cpython-38.pyc
create mode 100644 python/src/pywy/operators/__pycache__/unary.cpython-38.pyc
create mode 100644 python/src/pywy/operators/binary.py
create mode 100644
python/src/pywy/platforms/__pycache__/__init__.cpython-38.pyc
create mode 100644 python/src/pywy/platforms/__pycache__/java.cpython-38.pyc
create mode 100644 python/src/pywy/platforms/__pycache__/spark.cpython-38.pyc
copy python/src/pywy/platforms/{python/platform.py => java.py} (85%)
copy python/src/pywy/platforms/{python/platform.py => python.py} (92%)
copy python/src/pywy/platforms/{python/platform.py => spark.py} (84%)
rename python/src/pywy/platforms/{python/platform.py => tensorflow.py} (82%)
create mode 100644 python/src/pywy/tests/filter_test.py
rename python/{old_code/config/pywayang_config.ini =>
src/pywy/tests/json_test.py} (50%)
delete mode 100644 python/src/pywy/tests/resources/10e0MB.input
create mode 100644 python/src/pywy/tests/test.py
create mode 100644 python/src/pywy/tests/word_count.py
create mode 100644
wayang-api/wayang-api-json/src/main/resources/context-plan-a.json
create mode 100644
wayang-api/wayang-api-json/src/main/scala/operatorfromjson/ContextFromJson.scala
create mode 100644
wayang-api/wayang-api-json/src/main/scala/operatorfromjson/PlanFromJson.scala
create mode 100644
wayang-api/wayang-api-json/src/main/scala/operatorfromjson/binary/DLTrainingOperatorFromJson.scala
copy
wayang-api/wayang-api-json/src/main/scala/operatorfromjson/binary/{CoGroupOperatorFromJson.scala
=> PredictOperatorFromJson.scala} (70%)
copy
wayang-api/wayang-api-json/src/main/scala/parserutil/{ParseOperatorsFromJson.scala
=> ParsePlanFromJson.scala} (78%)
delete mode 100644
wayang-api/wayang-api-json/src/main/scala/parserutil/ParseScalaFromString.scala
copy wayang-api/wayang-api-json/src/main/scala/parserutil/{MyTypeName.scala =>
ParsingErrors.scala} (69%)
copy wayang-api/wayang-api-json/src/main/scala/parserutil/{MyTypeName.scala =>
SerializableIterable.scala} (58%)
copy wayang-api/wayang-api-json/src/main/scala/parserutil/{MyTypeName.scala =>
SerializableLambda.scala} (53%)
copy wayang-api/wayang-api-json/src/main/scala/parserutil/{MyTypeName.scala =>
SerializableLambda2.scala} (54%)
delete mode 100644
wayang-api/wayang-api-json/src/main/scala/springboot/SpringBootApplication.scala
delete mode 100644
wayang-api/wayang-api-json/src/main/scala/springboot/restapi/WayangApiJson.scala
create mode 100644
wayang-api/wayang-api-python/src/main/java/org/apache/wayang/api/python/function/WrappedFlatMapDescriptor.java
copy
wayang-api/wayang-api-python/src/main/java/org/apache/wayang/api/python/function/{PythonFunctionWrapper.java
=> WrappedMapPartitionsDescriptor.java} (54%)
create mode 100644
wayang-api/wayang-api-python/src/main/java/org/apache/wayang/api/python/function/WrappedPredicateDescriptor.java
rename
wayang-api/wayang-api-python/src/main/java/org/apache/wayang/api/python/function/{PythonUDF.java
=> WrappedPythonFunction.java} (58%)
create mode 100644
wayang-api/wayang-api-python/src/main/java/org/apache/wayang/api/python/function/WrappedReduceDescriptor.java
rename
wayang-api/wayang-api-python/src/main/java/org/apache/wayang/api/python/function/{PythonFunctionWrapper.java
=> WrappedTransformationDescriptor.java} (51%)
delete mode 100644
wayang-api/wayang-api-rest/src/main/java/org/apache/wayang/api/rest/server/spring/decoder/WayangPlanBuilder.java
delete mode 100644
wayang-api/wayang-api-rest/src/main/java/org/apache/wayang/api/rest/server/spring/general/WayangController.java
create mode 100644 wayang-api/wayang-api-scala-java/README.md
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/MultiContext.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/MultiContextDataQuanta.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/MultiContextPlanBuilder.scala
copy
wayang-api/{wayang-api-json/src/main/scala/operatorfromjson/binary/IntersectOperatorFromJson.scala
=>
wayang-api-scala-java/src/main/scala/org/apache/wayang/api/async/DataQuantaAsyncResult.scala}
(50%)
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/async/DataQuantaImplicits.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/async/Main.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/async/PlanBuilderImplicits.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/async/package.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/SerializationUtils.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/TempFileUtils.scala
copy
wayang-api/{wayang-api-python/src/main/java/org/apache/wayang/api/python/function/PythonCode.java
=>
wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/GenericSerializableDeserializer.scala}
(55%)
copy
wayang-api/{wayang-api-python/src/main/java/org/apache/wayang/api/python/function/PythonCode.java
=>
wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/GenericSerializableSerializer.scala}
(54%)
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/MultiContextDeserializer.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/MultiContextSerializer.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/OperatorDeserializer.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/PlatformDeserializer.scala
rename
wayang-api/{wayang-api-python/src/main/java/org/apache/wayang/api/python/function/PythonCode.java
=>
wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/customserializers/PlatformSerializer.scala}
(63%)
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/ConfigurationAndContextMixIns.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/DataTypeMixIns.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/DescriptorMixIns.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/EstimatorMixIns.scala
rename
wayang-api/{wayang-api-python/src/test/java/org/apache/wayang/api/python/PythonAPITest.java
=>
wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/IgnoreLoggerMixIn.scala}
(78%)
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/OperatorMixIns.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/ProviderMixIns.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/serialization/mixins/SlotMixIns.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/main/scala/org/apache/wayang/api/util/NDimArray.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/test/scala/org/apache/wayang/api/serialization/OperatorSerializationTests.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/test/scala/org/apache/wayang/api/serialization/OtherSerializationTests.scala
create mode 100644
wayang-api/wayang-api-scala-java/src/test/scala/org/apache/wayang/api/serialization/SerializationTestBase.scala
create mode 100644 wayang-assembly/.gitignore
create mode 100644 wayang-assembly/.rat-excludes
create mode 100644 wayang-assembly/dependency-reduced-pom.xml
create mode 100644
wayang-benchmark/src/main/scala/org/apache/wayang/async/apps/WordCount.scala
rename wayang-api/wayang-api-json/src/main/scala/parserutil/MyTypeName.scala
=>
wayang-benchmark/src/main/scala/org/apache/wayang/multicontext/apps/package.scala
(53%)
create mode 100644
wayang-benchmark/src/main/scala/org/apache/wayang/multicontext/apps/tpch/Query1.scala
create mode 100644
wayang-benchmark/src/main/scala/org/apache/wayang/multicontext/apps/wordcount/WordCount.scala
create mode 100644
wayang-benchmark/src/main/scala/org/apache/wayang/multicontext/apps/wordcount/WordCountCombineEach.scala
create mode 100644
wayang-benchmark/src/main/scala/org/apache/wayang/multicontext/apps/wordcount/WordCountWithMerge.scala
create mode 100644
wayang-benchmark/src/main/scala/org/apache/wayang/multicontext/apps/wordcount/WordCountWithTargetPlatforms.scala
create mode 100644
wayang-commons/wayang-basic/src/main/java/org/apache/wayang/basic/operators/PyWayangReduceByOperator.java
