Raphael, you are correct it doesn’t exist in the current version of Fey. We should put together a proposal for Fey Cluster and discuss this on the reflector so it become part of the Fey roadmap. Barbara do you want to put together an initial skeleton proposal so it can be discussed here and we can all help flesh it out? Once we have community consensus we can begin to move this to implementation. Hopefully, we can have a number of contributors help in the development of this. Thoughts?
-Tony > On May 26, 2017, at 12:18 PM, Rafael Weingärtner > <[email protected]> wrote: > > Thank you again for the reply Barbara, > > Let’s see if I understood everything that we have discussed so far; Fey does > only the recovery of agents that die within the same JVM (e.g. some > unexpected exception that breaks the life cycle of an agent/performer). > > > Your description of cluster is what I am looking for. To make sure I > understood everything so far; the cluster feature you described does not > exist yet, right? > > > On Fri, May 26, 2017 at 2:14 PM, Barbara Malta Gomes > <[email protected] <mailto:[email protected]>> wrote: > Hi Rafael, > > The orchestrations running on Fey are fault-tolerant in the sense that Fey > makes sure that if the actor in the same JVM dies, it gets restarted a > certain number of times, but Fey has no implementation of fail-over, in case > the JVM crashes. > > About what is a cluster in Fey Jargon, in my view, I think a Fey Cluster is > a distributed system where we have Fey instances running in different nodes > (slaves) and we have a Fey Master node that is responsible for the > distribution and managing the orchestrations, not limiting the scope of an > orchestration to only one JVM. In that case we would need to use Akka Remote > actors because the performers of an orchestration could be running in > different nodes. In this context we would have a fail-over mechanism in which > the Fey master node would redistribute the performers running on the JVM that > crashed to other Fey nodes. > > Regards > > On Fri, May 26, 2017 at 8:07 AM, Rafael Weingärtner > <[email protected] <mailto:[email protected]>> wrote: > Thanks for the reply Tony. From your answer, I am assuming that Fey does not > have the ability to be fault tolerant. This means, using more than one > runtime to host/execute agents, managing the fail-over in a transparent > manner. > > Before we start discussing a feature suggestion, what is a cluster in Fey > Jargon? > > Also, in Fey's Github README file I read the following: "Remote invocation of > actors make it possible for Orchestrations running in one location to > interact with Orchestrations running in another where services not available > locally can be accessed". When I read that I felt that I could invoke remote > agents (performers in different runtime) in a transparent manner. By > transparent here I mean, sending a message to an agent using an identifier, > and then the platform (Fey) takes care of locating the agent and actually > delivering the message. > > On Thu, May 25, 2017 at 5:22 PM, Tony Faustini <[email protected] > <mailto:[email protected]>> wrote: > Hi Rafael, good question what is meant by a Fey cluster? The Fey that I can > download today runs at on single server (say a Raspberry Pi at the low end) > and I would like also to run Fey at the Cluster level (say on a Mesos > cluster). My use case of has Fey running standalone on micro-servers that are > connected to the cloud. At the cloud I am running a Mess cluster and I want > to replace some existing technology running on Mesos with a version of Fey > that will run across that Mesos Cluster. This is one meaning of running Fey > as a cluster that I would like to see. I would like to see my performers with > upwards and downwards scalability over a cluster. > > Just for clarification my use case doesn’t look at the micro-servers and the > cloud Mesos cluster as one large network on which I want to run a single > instance of clustered Fey. It’s a simpler use case in which each > micros-server runs a standalone version of Fey and on my Mesos cluster I > need a version of Fey on the iota roadmap that will allow me to distribute my > orchestrations/performers across my Mesos cluster. Since Akka is part of the > SMACK stack (Spark, Mesos, Akka, Cassandra and Kafka) there may already be a > way of running Fey on a Mesos cluster. I would like to see the Fey cluster > version on the iota dev roadmap. > > There might be some interesting use cases in which Akka remoting might be > used. I would love to hear about them and in general other use cases that > folks have in mind for Fey. > > -Tony > > >> On May 25, 2017, at 11:58 AM, Rafael Weingärtner >> <[email protected] <mailto:[email protected]>> wrote: >> >> Thanks for the feedback Barbara. Lucas is in route today, so I am answering >> on his behalf. >> >> >> I think our confusion is with this expression “cluster context”. >> What do you (Fey) see as a cluster? Only a group of performers (agents) in >> the same runtime (JVM)? >> >> Also, about these security issues, are you talking about the security >> (confidentiality and integrity) of data transmitted from one performer to >> the other remotely? >> Regarding performers being unreachable; this might be the case if they are >> inside a NAT. However, in the IoT space, the idea is to plug everything >> directly to the Internet using IPv6; so, this should not be a problem, >> right? At least that is how I see this expression; I do not work with IoT, >> so I am probably wrong here. >> >> Having said that, our necessity is not in the IoT space, but still, Fey’s >> design fits quite well our system’s requirements. The only point is that we >> need our agents/performs to be fault-tolerant in case their runtime crashes. >> Therefore, redundancy for the performers (agents) platform is a must. >> >> If this is interesting for Fey as a community we could start working >> together on this matter. >> >> On Thu, May 25, 2017 at 1:25 PM, Barbara Malta Gomes >> <[email protected] <mailto:[email protected]>> wrote: >> Hi Lucas, >> >> Unfortunately Fey does not support remote connection between performers. >> >> Fey is not meant to be a connectivity protocol like MQTT or other messaging >> framework. >> The reason we do not implemented remote actors outside of a cluster context >> is because of security and also in most use cases the Performers will be >> unreachable from the "outside" (in a intranet). >> >> What we have today is an MQTT publisher Performer on one side and a MQTT >> subscriber in the other side. >> >> But we can definitely discuss more about implementing this functionality on >> Fey. >> >> Regards, >> >> >> On Wed, May 24, 2017 at 12:55 PM, Lucas Berri Cristofolini >> <[email protected] <mailto:[email protected]>> wrote: >> Both, actually... >> >> I suppose clusterization for fault tolerance is actually a more immediate >> concern, but the ability to run performers remotely is very interesting as >> well :) >> >> On May 24, 2017 4:47 PM, "Rafael Weingärtner" <[email protected] >> <mailto:[email protected]>> wrote: >> Do you want only peer-peer communication? Or do you want clustering of >> agents for fault-tolerant systems? >> http://doc.akka.io/docs/akka/snapshot/java/common/cluster.html >> <http://doc.akka.io/docs/akka/snapshot/java/common/cluster.html> >> >> On Wed, May 24, 2017 at 3:27 PM, Lucas Berri Cristofolini >> <[email protected] <mailto:[email protected]>> wrote: >> Yup, that worked just fine! >> >> I wrote the example you showed at ApcheCon in Java to test it out, and I was >> now wondering if it is possible to run the two performers in that ensemble >> in separate hosts. I suppose this would be akin to Akka's 'remoting' >> (http://doc.akka.io/docs/akka/snapshot/java/remoting.html >> <http://doc.akka.io/docs/akka/snapshot/java/remoting.html>). >> >> I saw some of the examples in GitHub but it isn't quite clear to me what I >> need to set in order for the performers to be able to communicate in that >> scenario. Are there any settings I can specify in the orchestration json to >> point to the remote host that would be running the other performer? >> >> >> PS: Thanks for the quick reply yesterday! >> >> On Tue, May 23, 2017 at 3:35 PM, Barbara Malta Gomes >> <[email protected] <mailto:[email protected]>> wrote: >> Hi Lucas, >> >> Good to hear that you are trying out iota. >> >> Sure, you can definitely write the performers in java. >> You just have to do like you said, add the Fey .jar as a dependency, >> implement a Performer that inherits from the FeyGenericActor and override >> the functions. >> >> Once you have done that, generate the .jar for the performer and use it on >> Fey. >> >> Let us know if you have any problems. >> >> Regards, >> >> On Tue, May 23, 2017 at 11:27 AM, Lucas Berri Cristofolini >> <[email protected] <mailto:[email protected]>> wrote: >> Hello, Iota community! :) >> >> One of my colleagues watched Barbara's talk at ApacheCon 2017 in Miami, and >> we were really interested in the way the Fey Engine works. >> >> I've been wanting to try it out since then but only got around to it today, >> and after going through the GitHub page for the project and Barbara's >> slides, I'm wondering if it is possible to write the performers in Java, >> since the Engine seems to take compiled .jars. >> >> If that's possible, how would one go about doing it? Is it just a matter of >> adding the fey engine as a dependency and overriding the right functions? >> >> Cheers! >> Lucas >> >> >> >> -- >> Barbara Gomes >> Computer Engineer >> San Jose, CA >> >> >> >> >> -- >> Rafael Weingärtner >> >> >> >> -- >> Barbara Gomes >> Computer Engineer >> San Jose, CA >> >> >> >> -- >> Rafael Weingärtner > > > > > -- > Rafael Weingärtner > > > > -- > Barbara Gomes > Computer Engineer > San Jose, CA > > > > -- > Rafael Weingärtner
