Thank you for all the feedback you've given, Barbara and Tony! The more I play around with Fey, the more I think it suits our project very well, even if it isn't exactly IoT focused (although I suppose it does share some concepts).
While the clustering for fault tolerance we were discussing isn't immediately necessary for us, it is on our roadmap and we find it essential for our project's completion. Having said that, we'd be absolutely willing to work together with you guys to bring that feature to fey. So how do you guys want to go about this? I don't know if Barbara has already started working on a design, if so we can discuss it with you guys and later help implement it. If not, maybe we could propose something ourselves and have the community discuss over it? Let me know what you think! On Fri, May 26, 2017 at 5:19 PM, Barbara Malta Gomes < [email protected]> wrote: > > Yes for both of your statement. > > I think it would be very valuable to start thinking and designing the > cluster implementation for Fey > > On Fri, May 26, 2017 at 12:18 PM, Rafael Weingärtner < > [email protected]> wrote: > >> Thank you again for the reply Barbara, >> >> Let’s see if I understood everything that we have discussed so far; Fey >> does only the recovery of agents that die within the same JVM (e.g. some >> unexpected exception that breaks the life cycle of an agent/performer). >> >> >> >> Your description of cluster is what I am looking for. To make sure I >> understood everything so far; the cluster feature you described does not >> exist yet, right? >> >> On Fri, May 26, 2017 at 2:14 PM, Barbara Malta Gomes < >> [email protected]> wrote: >> >>> Hi Rafael, >>> >>> The orchestrations running on Fey are fault-tolerant in the sense that >>> Fey makes sure that if the actor in the same JVM dies, it gets restarted a >>> certain number of times, but Fey has no implementation of fail-over, in >>> case the JVM crashes. >>> >>> About *what is a cluster in Fey Jargon*, in my view, I think a Fey >>> Cluster is a distributed system where we have Fey instances running in >>> different nodes (slaves) and we have a Fey Master node that is responsible >>> for the distribution and managing the orchestrations, not limiting the >>> scope of an orchestration to only one JVM. In that case we would need to >>> use Akka Remote actors because the performers of an orchestration could be >>> running in different nodes. In this context we would have a fail-over >>> mechanism in which the Fey master node would redistribute the performers >>> running on the JVM that crashed to other Fey nodes. >>> >>> Regards >>> >>> On Fri, May 26, 2017 at 8:07 AM, Rafael Weingärtner < >>> [email protected]> wrote: >>> >>>> Thanks for the reply Tony. From your answer, I am assuming that Fey >>>> does not have the ability to be fault tolerant. This means, using more than >>>> one runtime to host/execute agents, managing the fail-over in a transparent >>>> manner. >>>> >>>> Before we start discussing a feature suggestion, what is a cluster in >>>> Fey Jargon? >>>> >>>> Also, in Fey's Github README file I read the following: "Remote >>>> invocation of actors make it possible for Orchestrations running in one >>>> location to interact with Orchestrations running in another where services >>>> not available locally can be accessed". When I read that I felt that I >>>> could invoke remote agents (performers in different runtime) in a >>>> transparent manner. By transparent here I mean, sending a message to an >>>> agent using an identifier, and then the platform (Fey) takes care of >>>> locating the agent and actually delivering the message. >>>> >>>> On Thu, May 25, 2017 at 5:22 PM, Tony Faustini <[email protected]> wrote: >>>> >>>>> Hi Rafael, good question what is meant by a Fey cluster? The Fey that >>>>> I can download today runs at on single server (say a Raspberry Pi at the >>>>> low end) and I would like also to run Fey at the Cluster level (say on a >>>>> Mesos cluster). My use case of has Fey running standalone on micro-servers >>>>> that are connected to the cloud. At the cloud I am running a Mess cluster >>>>> and I want to replace some existing technology running on Mesos with a >>>>> version of Fey that will run across that Mesos Cluster. This is one >>>>> meaning >>>>> of running Fey as a cluster that I would like to see. I would like to see >>>>> my performers with upwards and downwards scalability over a cluster. >>>>> >>>>> Just for clarification my use case doesn’t look at the micro-servers >>>>> and the cloud Mesos cluster as one large network on which I want to run a >>>>> single instance of clustered Fey. It’s a simpler use case in which each >>>>> micros-server runs a standalone version of Fey and on my Mesos cluster I >>>>> need a version of Fey on the iota roadmap that will allow me to >>>>> distribute >>>>> my orchestrations/performers across my Mesos cluster. Since Akka is part >>>>> of >>>>> the SMACK stack (Spark, Mesos, Akka, Cassandra and Kafka) there may >>>>> already >>>>> be a way of running Fey on a Mesos cluster. I would like to see the Fey >>>>> cluster version on the iota dev roadmap. >>>>> >>>>> There might be some interesting use cases in which Akka remoting might >>>>> be used. I would love to hear about them and in general other use cases >>>>> that folks have in mind for Fey. >>>>> >>>>> -Tony >>>>> >>>>> >>>>> On May 25, 2017, at 11:58 AM, Rafael Weingärtner < >>>>> [email protected]> wrote: >>>>> >>>>> Thanks for the feedback Barbara. Lucas is in route today, so I am >>>>> answering on his behalf. >>>>> >>>>> I think our confusion is with this expression “cluster context”. >>>>> What do you (Fey) see as a cluster? Only a group of performers >>>>> (agents) in the same runtime (JVM)? >>>>> >>>>> Also, about these security issues, are you talking about the security >>>>> (confidentiality and integrity) of data transmitted from one performer to >>>>> the other remotely? >>>>> Regarding performers being unreachable; this might be the case if they >>>>> are inside a NAT. However, in the IoT space, the idea is to plug >>>>> everything >>>>> directly to the Internet using IPv6; so, this should not be a problem, >>>>> right? At least that is how I see this expression; I do not work with IoT, >>>>> so I am probably wrong here. >>>>> >>>>> Having said that, our necessity is not in the IoT space, but still, >>>>> Fey’s design fits quite well our system’s requirements. The only point is >>>>> that we need our agents/performs to be fault-tolerant in case their >>>>> runtime >>>>> crashes. Therefore, redundancy for the performers (agents) platform is a >>>>> must. >>>>> >>>>> If this is interesting for Fey as a community we could start working >>>>> together on this matter. >>>>> >>>>> On Thu, May 25, 2017 at 1:25 PM, Barbara Malta Gomes < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Lucas, >>>>>> >>>>>> Unfortunately Fey does not support remote connection between >>>>>> performers. >>>>>> >>>>>> Fey is not meant to be a connectivity protocol like MQTT or other >>>>>> messaging framework. >>>>>> The reason we do not implemented remote actors outside of a cluster >>>>>> context is because of security and also in most use cases the Performers >>>>>> will be unreachable from the "outside" (in a intranet). >>>>>> >>>>>> What we have today is an MQTT publisher Performer on one side and a >>>>>> MQTT subscriber in the other side. >>>>>> >>>>>> But we can definitely discuss more about implementing this >>>>>> functionality on Fey. >>>>>> >>>>>> Regards, >>>>>> >>>>>> >>>>>> On Wed, May 24, 2017 at 12:55 PM, Lucas Berri Cristofolini < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Both, actually... >>>>>>> >>>>>>> I suppose clusterization for fault tolerance is actually a more >>>>>>> immediate concern, but the ability to run performers remotely is very >>>>>>> interesting as well :) >>>>>>> >>>>>>> On May 24, 2017 4:47 PM, "Rafael Weingärtner" < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Do you want only peer-peer communication? Or do you want clustering >>>>>>>> of agents for fault-tolerant systems? >>>>>>>> http://doc.akka.io/docs/akka/snapshot/java/common/cluster.html >>>>>>>> >>>>>>>> On Wed, May 24, 2017 at 3:27 PM, Lucas Berri Cristofolini < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Yup, that worked just fine! >>>>>>>>> >>>>>>>>> I wrote the example you showed at ApcheCon in Java to test it out, >>>>>>>>> and I was now wondering if it is possible to run the two performers >>>>>>>>> in that >>>>>>>>> ensemble in separate hosts. I suppose this would be akin to Akka's >>>>>>>>> 'remoting' (http://doc.akka.io/docs/akka/ >>>>>>>>> snapshot/java/remoting.html). >>>>>>>>> >>>>>>>>> I saw some of the examples in GitHub but it isn't quite clear to >>>>>>>>> me what I need to set in order for the performers to be able to >>>>>>>>> communicate >>>>>>>>> in that scenario. Are there any settings I can specify in the >>>>>>>>> orchestration >>>>>>>>> json to point to the remote host that would be running the other >>>>>>>>> performer? >>>>>>>>> >>>>>>>>> >>>>>>>>> PS: Thanks for the quick reply yesterday! >>>>>>>>> >>>>>>>>> On Tue, May 23, 2017 at 3:35 PM, Barbara Malta Gomes < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Lucas, >>>>>>>>>> >>>>>>>>>> Good to hear that you are trying out iota. >>>>>>>>>> >>>>>>>>>> Sure, you can definitely write the performers in java. >>>>>>>>>> You just have to do like you said, add the Fey .jar as a >>>>>>>>>> dependency, implement a Performer that inherits from the >>>>>>>>>> FeyGenericActor >>>>>>>>>> and override the functions. >>>>>>>>>> >>>>>>>>>> Once you have done that, generate the .jar for the performer and >>>>>>>>>> use it on Fey. >>>>>>>>>> >>>>>>>>>> Let us know if you have any problems. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> On Tue, May 23, 2017 at 11:27 AM, Lucas Berri Cristofolini < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hello, Iota community! :) >>>>>>>>>>> >>>>>>>>>>> One of my colleagues watched Barbara's talk at ApacheCon 2017 in >>>>>>>>>>> Miami, and we were really interested in the way the Fey Engine >>>>>>>>>>> works. >>>>>>>>>>> >>>>>>>>>>> I've been wanting to try it out since then but only got around >>>>>>>>>>> to it today, and after going through the GitHub page for the >>>>>>>>>>> project and >>>>>>>>>>> Barbara's slides, I'm wondering if it is possible to write the >>>>>>>>>>> performers >>>>>>>>>>> in Java, since the Engine seems to take compiled .jars. >>>>>>>>>>> >>>>>>>>>>> If that's possible, how would one go about doing it? Is it just >>>>>>>>>>> a matter of adding the fey engine as a dependency and overriding >>>>>>>>>>> the right >>>>>>>>>>> functions? >>>>>>>>>>> >>>>>>>>>>> Cheers! >>>>>>>>>>> Lucas >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Barbara Gomes >>>>>>>>>> Computer Engineer >>>>>>>>>> San Jose, CA >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Rafael Weingärtner >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Barbara Gomes >>>>>> Computer Engineer >>>>>> San Jose, CA >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Rafael Weingärtner >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Rafael Weingärtner >>>> >>> >>> >>> >>> -- >>> Barbara Gomes >>> Computer Engineer >>> San Jose, CA >>> >> >> >> >> -- >> Rafael Weingärtner >> > > > > -- > Barbara Gomes > Computer Engineer > San Jose, CA >
