Hi all,
I'm doing this project for GSOC 2019. This email is to update the
initiative progress of the project and this thread will be used in future
for updating the progress too.
First of all here [0] is a basic POC I did.
After that I found out that there is already a HDFS connector for hazelcast
jet[1]. Then I looked into using this. But encountered with some package
conflicts('org.apache.hadoop.mapreduce', and 'org.apache.hadoop.mapred').
Therefore finally after further discussions with mentors, it was decided to
implement custom source and sink as of this documentation[2].
Now i'm in the process of implementing this. Your ideas and feedback are
appreciated.
[0]
https://github.com/LahiruJayasekara/gora/blob/poc-hazelcast-jet/gora-tutorial/src/main/java/org/apache/gora/tutorial/log/HazelcastJetPOC.java
[1] https://docs.hazelcast.org/docs/jet/0.7/manual/#hdfs
[2]
https://docs.hazelcast.org/docs/jet/3.0/manual/#custom-source-sink-vertex
Thanks and Regards
--
Lahiru Jayasekara
Batch 15
Faculty of Information Technology
University of Moratuwa
0716492170
