Hi Julian, Thanks for pointing to NiFi, I'll have a look at it. I used edgent at that time since it was simple enough to get a custom topology up and running without any added bloat.
Regarding the dataset, it is a series of unrelated events and definitely partitioning is possible. But I was not worried about that aspect, since I was focusing on placement of the topology and not on improving the performance through parallelism of some sort. Is it why you asked that? I'll start a new thread on the fault tolerance aspects. It should be interesting to understand others' view on it. /Gayashan On Tue, Nov 6, 2018 at 8:58 PM Julian Feinauer <j.feina...@pragmaticminds.de> wrote: > Hey Gayashan, > > thanks for the insights. > Sounds very interesting, another interesting tool for that could be Apache > NiFi / MiNiFy which has this "partitioning already in its core (but better > keep here ; ) > Are these datasets timeseries of related or rather unrelated events, so > does order matter, is partitioning possible? > > And regarding Fault Tolerance, that is a wide and very interesting field, > but I agree, better start a new thread for that and then discuss it there > in dept. > > Julian > > Am 06.11.18, 10:51 schrieb "Gayashan Amarasinghe" < > gayashan.amarasin...@gmail.com>: > > HI Julian, > > Thank you for your kind reply and understanding. > > I've been working on sharing the work load of distributed stream > processing > between the edge and the cloud. So what I have been doing now is > running > edgent on both the edge and the cloud. To be specific I was running a > part > of the edgent streaming topology on the edge where my event sources are > located, do some processing there, send the processed stream to the > cloud > and run the final part of the streaming topology on the cloud to > complete > the processing there. So basically a big edgent topology partitioned > between the edge and the cloud. Does that make sense? > > I'm currently looking at my next idea and thinking of fault tolerance > aspects that could be required. What are your thoughts on fault > tolerance > requirements of edgent from an industry/real user perspective? (May be > I > should bring this discussion to a new thread on the list.) > > Thanks! > > Best, > Gayashan > > > On Tue, Nov 6, 2018 at 6:47 PM Julian Feinauer < > j.feina...@pragmaticminds.de> > wrote: > > > Hi Nuwan, > > > > it's a pleasure to hear from you! > > I didn’t know that there is so much usage of edge processing in > academia, > > that’s pretty cool to hear. > > As I tried to dig into the Eclipse IoT Eco System the last time, > it's good > > to hear that we have people on the list that are familiar with > Mozillas > > Gateway! > > > > And, to be honest, it would be great if you could find some time to > > contribute documentation or media as this is something programmers > tend to > > skip __ > > > > Best > > Julian > > > > PS.: > > @Nuwan and @Gayashan: > > Would it be possible for you to share some details about your use > cases > > (where you used edgent, or at least considered to use edgent) with > us? > > I would be really interested in it. > > > > Am 06.11.18, 04:12 schrieb "Nuwan Jayawardene" <nsga...@gmail.com>: > > > > Hi All, > > > > I'm an undergrad following a CS degree in Sri Lanka. > > I came across Edgent when doing research for my final year > thesis on > > Fog > > Networks and IoT. > > Even though Edgent isn't in my in project scope (it was > overkill, I > > went > > with Mozilla's Things Gateway) for the time being, I had definite > > plans to > > include it in my implementation afterwards. The Open Source > middleware > > vendor I'm working with was also extremely interested to see how > > Edgent can > > be integrated into their platform. > > > > I've never mailed to this mailing list or committed to this > project > > before, > > but I'm doing so now as I want to see Edgent grow into a mature > Apache > > project. > > While it's difficult for me to delegate a lot of time at the > moment, I > > will > > contribute to the best of my ability in other aspects like > > documentation > > and media. > > Hope to get a response soon. > > > > Thanks and best regards. > > > > On Tue, Nov 6, 2018 at 7:18 AM Gayashan Amarasinghe < > > gayashan.amarasin...@gmail.com> wrote: > > > > > Hi all, > > > > > > I am a phd student at University of Melbourne who is doing my > > research in > > > Edge computing + Distributed Stream Processing. I have been > following > > > edgent (and I have followed Christofer to plc4x as well since > I have > > seen > > > him mentioning it here the work that he has done) for some > time now. > > In > > > fact I have sent couple of emails some time back but haven't > been > > able to > > > do anything significant. I have also used edgent in one of my > ongoing > > > research work even though it has not been published yet. I > thought > > I'd send > > > an email after seeing this thread. > > > > > > To simply put, I would like to help with the rebooting edgent, > I > > think I > > > would be able to use it in a mutually beneficial way such that > I > > could use > > > edgent to improve my research and get involved with edgent and > > edgent may > > > get some features/bug fixes/etc as a result. But I have to be > honest > > that > > > being a phd student, I may not be able to be very active. But > I am > > willing > > > to do as much as possible in the free time I have. Let me know > what > > you > > > think and may be a way we can work together. > > > > > > Thank you. > > > > > > Best, > > > Gayashan > > > > > > > > > On Sun, Nov 4, 2018 at 9:09 PM Julian Feinauer < > > > j.feina...@pragmaticminds.de> > > > wrote: > > > > > > > Hey Justin, > > > > > > > > thanks for your response. > > > > Please don't take my words too literal (as I'm not (yet) > used to > > many of > > > > the ASF processes). > > > > It's good to hear that you are still onboard. For me as > rookie, > > it's good > > > > to hear that we have multiple seasoned ASF member onboard. > > > > > > > > Regarding my statement with the attic.. what I really meant > was to > > > > "finish" the project officially, because currently, its kind > of in > > a > > > zombie > > > > mode. > > > > > > > > But, I would definitely prefer a "revival" : ) > > > > > > > > Julian > > > > > > > > Am 03.11.18, 22:59 schrieb "Justin Mclean" < > > jus...@classsoftware.com>: > > > > > > > > Hi, > > > > > > > > > I think we (as we-that-are-on-the-list) have to decide > > whether we > > > > want to (re-)start the project, building a new PPMC > (currently we > > have > > > > effectively one with chris) and start to do some work in > edgent. > > > > > > > > As far as PPMC (and mentor) duties I’m still around. The > board > > likes > > > a > > > > minimum of 3 PPMC members, to graduate a minimal of 5 is > seen as a > > good > > > > number. > > > > > > > > > Or whether we decide (or formally I guess PPMC and the > > mentors > > > > decide) to move edgent to the attic. > > > > > > > > Podlings that don’t make it, don’t end up in the attic, > but > > the code > > > > could continue to live on github. > > > > > > > > Thanks, > > > > Justin > > > > > > > > > > > > > > > > > -- > > > > *Nuwan Jayawardene* > > Computer Science Undergraduate at Informatics Institute of > > Technology(IIT), > > Colombo > > Tel - +94 71 181 3299 > > https://www.linkedin.com/in/nuwanjaya/ > > https://medium.com/@nuwanjaya > > > > > > > > >
