This is an automated email from the ASF dual-hosted git repository.
zkaoudi pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-wayang.git
from 74b91763 Merge pull request #523 from mspruc/main
add 2193e49d Create ParquetSource
add 50934415 Update Record.java
add 17759ef8 Add readParquet to plan builders
add c97fa727 Add Java execution operator to ParquetSource
add 798ec683 Add SparkSession to executor
add 37cc8abc Add Spark execution operator to ParquetSource
add 0041654e Create WordCount example with parquet file
add dcd26d30 Update parquet word count example for projections
add 4d181516 Add setField and addField to Record
add 610aef83 Merge remote-tracking branch 'upstream/main'
add 6fa1a240 Merge latest changes
add 5c86222f Remove comment
add 0109c022 Merge branch 'main' into main
add d14fd248 Merge remote-tracking branch 'origin/main'
new 4df21216 Merge pull request #528 from aurb9/main
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/wayang/api/JavaPlanBuilder.scala | 10 +-
.../scala/org/apache/wayang/api/PlanBuilder.scala | 12 +-
.../wayang/apps/wordcount/WordCountParquet.java | 53 +++---
wayang-commons/wayang-basic/pom.xml | 5 +
.../java/org/apache/wayang/basic/data/Record.java | 27 ++++
.../wayang/basic/operators/ParquetSource.java | 178 ++++++++++++++++++++
wayang-platforms/wayang-java/pom.xml | 10 ++
.../org/apache/wayang/java/mapping/Mappings.java | 1 +
...ourceMapping.java => ParquetSourceMapping.java} | 15 +-
.../wayang/java/operators/JavaParquetSource.java | 180 +++++++++++++++++++++
.../main/resources/wayang-java-defaults.properties | 20 +++
.../wayang/spark/execution/SparkExecutor.java | 7 +
.../org/apache/wayang/spark/mapping/Mappings.java | 1 +
...ourceMapping.java => ParquetSourceMapping.java} | 11 +-
...TextFileSource.java => SparkParquetSource.java} | 58 ++++---
.../resources/wayang-spark-defaults.properties | 28 ++++
16 files changed, 558 insertions(+), 58 deletions(-)
copy guides/WordCount.java =>
wayang-benchmark/src/main/java/org/apache/wayang/apps/wordcount/WordCountParquet.java
(57%)
create mode 100644
wayang-commons/wayang-basic/src/main/java/org/apache/wayang/basic/operators/ParquetSource.java
copy
wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/mapping/{TextFileSourceMapping.java
=> ParquetSourceMapping.java} (80%)
create mode 100644
wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/operators/JavaParquetSource.java
copy
wayang-platforms/wayang-spark/src/main/java/org/apache/wayang/spark/mapping/{CollectionSourceMapping.java
=> ParquetSourceMapping.java} (82%)
copy
wayang-platforms/wayang-spark/src/main/java/org/apache/wayang/spark/operators/{SparkTextFileSource.java
=> SparkParquetSource.java} (66%)
