Hi Lucas, Glad to know that Fey suits your project.
Why don't you guys start working on the clustering design and we can move forward discussing about the details? I would like to see what you guys have in mind for Fey in "Cluster mode" Regards, On Mon, May 29, 2017 at 7:35 AM, Lucas Berri Cristofolini < [email protected]> wrote: > Thank you for all the feedback you've given, Barbara and Tony! > > The more I play around with Fey, the more I think it suits our project > very well, even if it isn't exactly IoT focused (although I suppose it does > share some concepts). > > While the clustering for fault tolerance we were discussing isn't > immediately necessary for us, it is on our roadmap and we find it essential > for our project's completion. Having said that, we'd be absolutely willing > to work together with you guys to bring that feature to fey. > > So how do you guys want to go about this? I don't know if Barbara has > already started working on a design, if so we can discuss it with you guys > and later help implement it. If not, maybe we could propose something > ourselves and have the community discuss over it? > > Let me know what you think! > > On Fri, May 26, 2017 at 5:19 PM, Barbara Malta Gomes < > [email protected]> wrote: > >> >> Yes for both of your statement. >> >> I think it would be very valuable to start thinking and designing the >> cluster implementation for Fey >> >> On Fri, May 26, 2017 at 12:18 PM, Rafael Weingärtner < >> [email protected]> wrote: >> >>> Thank you again for the reply Barbara, >>> >>> Let’s see if I understood everything that we have discussed so far; Fey >>> does only the recovery of agents that die within the same JVM (e.g. some >>> unexpected exception that breaks the life cycle of an agent/performer). >>> >>> >>> >>> Your description of cluster is what I am looking for. To make sure I >>> understood everything so far; the cluster feature you described does not >>> exist yet, right? >>> >>> On Fri, May 26, 2017 at 2:14 PM, Barbara Malta Gomes < >>> [email protected]> wrote: >>> >>>> Hi Rafael, >>>> >>>> The orchestrations running on Fey are fault-tolerant in the sense that >>>> Fey makes sure that if the actor in the same JVM dies, it gets restarted a >>>> certain number of times, but Fey has no implementation of fail-over, in >>>> case the JVM crashes. >>>> >>>> About *what is a cluster in Fey Jargon*, in my view, I think a Fey >>>> Cluster is a distributed system where we have Fey instances running in >>>> different nodes (slaves) and we have a Fey Master node that is responsible >>>> for the distribution and managing the orchestrations, not limiting the >>>> scope of an orchestration to only one JVM. In that case we would need to >>>> use Akka Remote actors because the performers of an orchestration could be >>>> running in different nodes. In this context we would have a fail-over >>>> mechanism in which the Fey master node would redistribute the performers >>>> running on the JVM that crashed to other Fey nodes. >>>> >>>> Regards >>>> >>>> On Fri, May 26, 2017 at 8:07 AM, Rafael Weingärtner < >>>> [email protected]> wrote: >>>> >>>>> Thanks for the reply Tony. From your answer, I am assuming that Fey >>>>> does not have the ability to be fault tolerant. This means, using more >>>>> than >>>>> one runtime to host/execute agents, managing the fail-over in a >>>>> transparent >>>>> manner. >>>>> >>>>> Before we start discussing a feature suggestion, what is a cluster in >>>>> Fey Jargon? >>>>> >>>>> Also, in Fey's Github README file I read the following: "Remote >>>>> invocation of actors make it possible for Orchestrations running in one >>>>> location to interact with Orchestrations running in another where services >>>>> not available locally can be accessed". When I read that I felt that I >>>>> could invoke remote agents (performers in different runtime) in a >>>>> transparent manner. By transparent here I mean, sending a message to an >>>>> agent using an identifier, and then the platform (Fey) takes care of >>>>> locating the agent and actually delivering the message. >>>>> >>>>> On Thu, May 25, 2017 at 5:22 PM, Tony Faustini <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Rafael, good question what is meant by a Fey cluster? The Fey that >>>>>> I can download today runs at on single server (say a Raspberry Pi at the >>>>>> low end) and I would like also to run Fey at the Cluster level (say on a >>>>>> Mesos cluster). My use case of has Fey running standalone on >>>>>> micro-servers >>>>>> that are connected to the cloud. At the cloud I am running a Mess cluster >>>>>> and I want to replace some existing technology running on Mesos with a >>>>>> version of Fey that will run across that Mesos Cluster. This is one >>>>>> meaning >>>>>> of running Fey as a cluster that I would like to see. I would like to see >>>>>> my performers with upwards and downwards scalability over a cluster. >>>>>> >>>>>> Just for clarification my use case doesn’t look at the micro-servers >>>>>> and the cloud Mesos cluster as one large network on which I want to run a >>>>>> single instance of clustered Fey. It’s a simpler use case in which each >>>>>> micros-server runs a standalone version of Fey and on my Mesos cluster I >>>>>> need a version of Fey on the iota roadmap that will allow me to >>>>>> distribute >>>>>> my orchestrations/performers across my Mesos cluster. Since Akka is part >>>>>> of >>>>>> the SMACK stack (Spark, Mesos, Akka, Cassandra and Kafka) there may >>>>>> already >>>>>> be a way of running Fey on a Mesos cluster. I would like to see the Fey >>>>>> cluster version on the iota dev roadmap. >>>>>> >>>>>> There might be some interesting use cases in which Akka remoting >>>>>> might be used. I would love to hear about them and in general other use >>>>>> cases that folks have in mind for Fey. >>>>>> >>>>>> -Tony >>>>>> >>>>>> >>>>>> On May 25, 2017, at 11:58 AM, Rafael Weingärtner < >>>>>> [email protected]> wrote: >>>>>> >>>>>> Thanks for the feedback Barbara. Lucas is in route today, so I am >>>>>> answering on his behalf. >>>>>> >>>>>> I think our confusion is with this expression “cluster context”. >>>>>> What do you (Fey) see as a cluster? Only a group of performers >>>>>> (agents) in the same runtime (JVM)? >>>>>> >>>>>> Also, about these security issues, are you talking about the security >>>>>> (confidentiality and integrity) of data transmitted from one performer to >>>>>> the other remotely? >>>>>> Regarding performers being unreachable; this might be the case if >>>>>> they are inside a NAT. However, in the IoT space, the idea is to plug >>>>>> everything directly to the Internet using IPv6; so, this should not be a >>>>>> problem, right? At least that is how I see this expression; I do not work >>>>>> with IoT, so I am probably wrong here. >>>>>> >>>>>> Having said that, our necessity is not in the IoT space, but still, >>>>>> Fey’s design fits quite well our system’s requirements. The only point is >>>>>> that we need our agents/performs to be fault-tolerant in case their >>>>>> runtime >>>>>> crashes. Therefore, redundancy for the performers (agents) platform is a >>>>>> must. >>>>>> >>>>>> If this is interesting for Fey as a community we could start working >>>>>> together on this matter. >>>>>> >>>>>> On Thu, May 25, 2017 at 1:25 PM, Barbara Malta Gomes < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Lucas, >>>>>>> >>>>>>> Unfortunately Fey does not support remote connection between >>>>>>> performers. >>>>>>> >>>>>>> Fey is not meant to be a connectivity protocol like MQTT or other >>>>>>> messaging framework. >>>>>>> The reason we do not implemented remote actors outside of a cluster >>>>>>> context is because of security and also in most use cases the Performers >>>>>>> will be unreachable from the "outside" (in a intranet). >>>>>>> >>>>>>> What we have today is an MQTT publisher Performer on one side and a >>>>>>> MQTT subscriber in the other side. >>>>>>> >>>>>>> But we can definitely discuss more about implementing this >>>>>>> functionality on Fey. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> >>>>>>> On Wed, May 24, 2017 at 12:55 PM, Lucas Berri Cristofolini < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Both, actually... >>>>>>>> >>>>>>>> I suppose clusterization for fault tolerance is actually a more >>>>>>>> immediate concern, but the ability to run performers remotely is very >>>>>>>> interesting as well :) >>>>>>>> >>>>>>>> On May 24, 2017 4:47 PM, "Rafael Weingärtner" < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Do you want only peer-peer communication? Or do you want >>>>>>>>> clustering of agents for fault-tolerant systems? >>>>>>>>> http://doc.akka.io/docs/akka/snapshot/java/common/cluster.html >>>>>>>>> >>>>>>>>> On Wed, May 24, 2017 at 3:27 PM, Lucas Berri Cristofolini < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Yup, that worked just fine! >>>>>>>>>> >>>>>>>>>> I wrote the example you showed at ApcheCon in Java to test it >>>>>>>>>> out, and I was now wondering if it is possible to run the two >>>>>>>>>> performers in >>>>>>>>>> that ensemble in separate hosts. I suppose this would be akin to >>>>>>>>>> Akka's >>>>>>>>>> 'remoting' (http://doc.akka.io/docs/akka/ >>>>>>>>>> snapshot/java/remoting.html). >>>>>>>>>> >>>>>>>>>> I saw some of the examples in GitHub but it isn't quite clear to >>>>>>>>>> me what I need to set in order for the performers to be able to >>>>>>>>>> communicate >>>>>>>>>> in that scenario. Are there any settings I can specify in the >>>>>>>>>> orchestration >>>>>>>>>> json to point to the remote host that would be running the other >>>>>>>>>> performer? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> PS: Thanks for the quick reply yesterday! >>>>>>>>>> >>>>>>>>>> On Tue, May 23, 2017 at 3:35 PM, Barbara Malta Gomes < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Lucas, >>>>>>>>>>> >>>>>>>>>>> Good to hear that you are trying out iota. >>>>>>>>>>> >>>>>>>>>>> Sure, you can definitely write the performers in java. >>>>>>>>>>> You just have to do like you said, add the Fey .jar as a >>>>>>>>>>> dependency, implement a Performer that inherits from the >>>>>>>>>>> FeyGenericActor >>>>>>>>>>> and override the functions. >>>>>>>>>>> >>>>>>>>>>> Once you have done that, generate the .jar for the performer and >>>>>>>>>>> use it on Fey. >>>>>>>>>>> >>>>>>>>>>> Let us know if you have any problems. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> On Tue, May 23, 2017 at 11:27 AM, Lucas Berri Cristofolini < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hello, Iota community! :) >>>>>>>>>>>> >>>>>>>>>>>> One of my colleagues watched Barbara's talk at ApacheCon 2017 >>>>>>>>>>>> in Miami, and we were really interested in the way the Fey Engine >>>>>>>>>>>> works. >>>>>>>>>>>> >>>>>>>>>>>> I've been wanting to try it out since then but only got around >>>>>>>>>>>> to it today, and after going through the GitHub page for the >>>>>>>>>>>> project and >>>>>>>>>>>> Barbara's slides, I'm wondering if it is possible to write the >>>>>>>>>>>> performers >>>>>>>>>>>> in Java, since the Engine seems to take compiled .jars. >>>>>>>>>>>> >>>>>>>>>>>> If that's possible, how would one go about doing it? Is it just >>>>>>>>>>>> a matter of adding the fey engine as a dependency and overriding >>>>>>>>>>>> the right >>>>>>>>>>>> functions? >>>>>>>>>>>> >>>>>>>>>>>> Cheers! >>>>>>>>>>>> Lucas >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Barbara Gomes >>>>>>>>>>> Computer Engineer >>>>>>>>>>> San Jose, CA >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Rafael Weingärtner >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Barbara Gomes >>>>>>> Computer Engineer >>>>>>> San Jose, CA >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Rafael Weingärtner >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Rafael Weingärtner >>>>> >>>> >>>> >>>> >>>> -- >>>> Barbara Gomes >>>> Computer Engineer >>>> San Jose, CA >>>> >>> >>> >>> >>> -- >>> Rafael Weingärtner >>> >> >> >> >> -- >> Barbara Gomes >> Computer Engineer >> San Jose, CA >> > > -- Barbara Gomes Computer Engineer San Jose, CA
