Hi Madhawa, Sorry for the late reply. Sure I'll try that and let you know. Thanks
On Tue, Apr 23, 2019 at 2:24 PM Madhawa Kasun Gunasekara < madhaw...@gmail.com> wrote: > Hi Lahiru, > > Good initiative. > It seems like we need to use hadoop-hdfs and hadoop common with version > 2.8.3. Try adding these dependencies, and exclude the old dependency from > the gora, or otherwise, we can do a version upgrade in gora for Hadoop. at > the moment we use Hadoop version 2.5.2 in gora but I prefer to upgrade the > Hadoop versions in the gora to 2.8.3. > > Recently hazelcast developers released 3.0 version, We can try that also. > > Thanks, > Madhawa > > > On Sat, Apr 6, 2019 at 9:53 PM Lahiru Jayasekera <mlpjayasek...@gmail.com> > wrote: > > > Hi all, > > I have come up with a poc for hazelcast jet execution engine support for > > gora. > > > > > https://github.com/LahiruJayasekara/gora/blob/poc-hazelcast-jet/gora-tutorial/src/main/java/org/apache/gora/tutorial/log/HazelcastJetPOC.java > > > > Here I have wrote a custom source for jet. What this does is read the > > AccessLog table created via LogManager example, and feed the PageView > > objects to jet. > > > > I have also looked into using hdfs connectors of jet. But I encountered a > > problem with conflicting hadoop versions. InputFormat class used in gora > is > > from pacjage 'org.apache.hadoop.mapreduce', but in jet it is ' > > org.apache.hadoop.mapred.InputFormat'. > > > > Anyway the proposed poc works as expected. Please add your suggestions > > here. > > > > Thanks and regards > > > > -- > > Lahiru Jayasekara > > Batch 15 > > Faculty of Information Technology > > University of Moratuwa > > 0716492170 > > > -- Lahiru Jayasekara Batch 15 Faculty of Information Technology University of Moratuwa 0716492170
