Hi, +1 for distribution of properties via S3, makes perfect sense. Perhaps abstracting behind an API so that any low latency globally distributed storage provider could be used.
Not sure about discovery. Although [0] described the AWS VM, it doesn't, without further validation describe if the Sling instance is running and available. Its perfectly possible for the VM to be in a running state, with no viable Sling instance running. I dont think that hard to achieve but it needs to be done to support the discovery use case. I think we are talking about instances running on independent repositories here, since if all instances share the same repository (ie are a Jackrabbit cluster), then the repository already has a mechanism of communicating running instances via the repository. Best Regards Ian On 12 May 2014 07:06, Carsten Ziegeler <cziege...@apache.org> wrote: > Hi Timotheé, > > yes I think this is valuable - the idea of the discovery API is to abstract > the discovery and if we can benefit in certain scenarios from already > available mechanism/information I think it makes totally sense to use that > instead of adding the same on top of it. > > Right now, the topology is formed of clusters containing instances - where > all instances in a cluster share the same repository, but instances in > different clusters use a different one. Is this kind of topology somehow > possible by using the AWS API? Or would all instances end up in a single > cluster? > > Regards > Carsten > > > 2014-05-11 18:54 GMT+02:00 Timothée Maret <timothee.ma...@gmail.com>: > >> Hi, >> >> I would like to discuss a potential implementation of the Sling Discovery >> APIs over an eventually consistent distributed storages such as AWS S3. >> Assuming the instances being part of the topology runs in AWS, then we >> could leverage AWS APIs and service in order to implement the Discovery >> mechanism. >> >> The discovery of instances could be implemented implicitely using EC2 REST >> API [0] without sending heartbeats, the properties for each instance could >> be stored in AWS S3 and distributed eventually, the leader election could >> be implemented with [1] or similar. >> >> The benefits (over Sling impl) would be >> * Arguably the highest availablity we can get from the environment >> * Reduced bandwith consumption (no hearthbeats) >> * Environment specific informations is implicitely distributed (local ip, >> external ip, hostname, region, etc.) >> >> Of course, it would bind the implementation to an environment (AWS in this >> case), however I believe we could apply the same mechanism to other >> eventually consistent storage. >> >> Wdyt ? Is this something that would be valuable for Sling ? >> >> Regards, >> >> Timothee >> >> [0] >> >> http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html >> [1] http://gsyc.es/~anto/papers/2007-dsn.pdf >> > > > > -- > Carsten Ziegeler > cziege...@apache.org
