Hi Hector looks good. Sling wont ship a Cassandra instance, for this project it will uses a Cassandra instance setup separately Last time I spun up Cassandra it for dev purposes it was just as easy as installing MySQL or PostgreSQL, so I think that fine.
If using Hector, I think it would be good to do everything in CQL and keep it all very simple and transparent. Remember the aim of the project is to prove that the ResourceProvider API can support Cassandra as a repository store, and that API is complete and usable all the way through to the latest security related APIs that have just been developed. This project is not an exercise in doing cool and complex things in Cassandra with automated ORM mapping that binds the code forever to one Cassandra API. Sound ok to you, or do you see problems ? Ian On 6 April 2013 16:55, Dishara Wijewardana <[email protected]> wrote: > Hi Ian, > > In Sling, are we going to ship native Cassandra instance with the sling > binary pack? ( is there already a native cassandra server starting at sling > server start up ?) > I am willing to use hector API to deal with Cassandra, as in most of > occasions around . Please let me know if there is any concerns on that > aspect ? > > > On Thu, Apr 4, 2013 at 1:04 AM, Dishara Wijewardana < > [email protected] > > wrote: > > > Hi Ian, > > > > On Tue, Apr 2, 2013 at 4:44 AM, Ian Boston <[email protected]> wrote: > > > >> On 2 April 2013 07:55, Dishara Wijewardana <[email protected]> > >> wrote: > >> > Hi Ian, > >> > Thanks for the explanation. Really helped me to get several stuff > >> cleared. > >> > > >> > On Mon, Apr 1, 2013 at 7:57 AM, Ian Boston <[email protected]> wrote: > >> > > >> >> Hi Dishara, > >> >> > >> >> You should not think about implementing the JCR API classes > >> >> (javax.jcr.*). You should think about implementing the Resource > >> >> API[1]. When you need to perform something that the ResourceAPI > doesnt > >> >> support (say in a servlet), then you can use the > >> >> Resource.adapTo(Class<?> clazz) method to adapt the resource into a > >> >> class that supports that operation. > >> >> > >> > > >> > Interesting.!! . If I got you correctly "adapTo" concept used to > achieve > >> > *kind of* dynamic multiple inheritance rather than extending a heavy > >> > abstract class. if that is so, this is a wonderful idea and I really > >> like > >> > the concept. > >> > >> Calling it inheritance might be going too far as the class isn't > >> necessarily the same, but in essence, yes. > >> > >> > > >> > > >> >> > >> >> eg: > >> >> The properties of the resource are exposed via ResourceMetadata [2], > >> >> but lets say you want to find the underlying Cassandra object to copy > >> >> it and create a child. > >> >> > >> >> So: CassandraResource implements Resource, which is what is returned > >> >> when the CassandraResourceProvider resolves a Resource. A client > >> >> should not bind to CassandraResource or mention it, since its almost > >> >> certainly an implementation and not an API. However, the Cassandra > >> >> bundle you have written also implements CassandraContent which has > >> >> CassandraContent.copy(String copyLocation) and > >> >> CassandraContent.createChild(String childName). > >> > > >> > To get to this you would do > >> >> CassandraResource.adaptTo(CassandraContent.class) > >> >> BTW, CassandraContent.class is an API exported by your bundle. > >> >> > >> > > >> > If I got you correctly, > >> > after CassandraResource.adaptTo(CassandraContent.class) I should be > >> able > >> > to call > >> > CassandraResource.copy(String copyLocation) > >> > and CassandraResource.createChild(String childName). > >> > > >> > Then I will have to write an "Adaptable" class to > >> > facilitate CassandraResource to deal with its corresponding cassandra > >> > nodes. > >> > In that sense I feel the best way to approach to this project when > >> > implementing is an bottom-up approach. We start from the "Adaptable" > >> class > >> > (given the fact that it should be expanded during implementation) and > >> then > >> > goes to the sling Resource wrapper layer which is > CassandraResourceImpl. > >> > >> > >> Resource extends Adaptable, so you dont get much choice there, just > >> implement the resource. > >> > >> > > >> > What do you think ? And may I know the expected scope of this project > by > >> > the community (just a potential one, during the implementation we can > >> > expand it as time permits) .i.e > >> > READ, > >> > READ with access control > >> > READ/WRITE with access control and etc. > >> > > >> > >> Yes, thats the idea. > >> It would be nicer to have 4 iterations to fit the GSoC timeline and do > >> 2 before half time, and 2 after. Perhaps the first half can also > > > > include spinning up Cassandra, getting client APIs working and getting > >> fully upto speed with that area of Sling. > >> > > > > +1 . Will make in to four iteration (generally 2 before midterm and 2 > > after mid term) and will include getting client API working for Cassandra > > and related sling stuff as a part of first two iterations (which means > > before mid term ). And to meet deadlines, should target more to be > > completed before mid term compared to the work scheduled after the mid > > term. Will make it count in project proposal time line schedule. And > thank > > you for the aforementioned feedback so that I can provide a solid > proposal > > in this year as well. > > Will post as soon as when I got update. Meanwhile I will build sling and > > getting familiar with the code base with the help of the sling > > documentation. > > > > > >> > >> > >> Ian > >> > > >> >> I hope that gives you an idea of how a CassandraResourceProvider and > >> >> associated implementation should work. If you havent already, and are > >> >> still interested you should read up on how Sling and OSGi works. > >> >> Trying the "Sling in 15 minutes"[3] is a good place to start. And > then > >> >> reading everything you can about Declarative Services in OSGi. The 2 > >> >> key things you are going to need to understand is how OSGi bundles > >> >> import and export packages and then how Declarative, declare services > >> >> that they depend on and declare services they implement, [4] is a > >> >> bunch of links. > >> >> > >> > > >> > Yes, I will go through the mentioned links. Specially like [3] which I > >> have > >> > not yet gone through. I also have some experience in using OSGi > >> importing > >> > exporting packages and using maven scr plugin in classes with "bind" > >> > "unbind" attributes to register and unregister OSGi services and etc. > >> > > >> > > >> >> HTH > >> >> Ian > >> >> > >> >> > >> >> 1 > >> >> > >> > http://sling.apache.org/apidocs/sling6/org/apache/sling/api/resource/Resource.html > >> >> 2 > >> >> > >> > http://sling.apache.org/apidocs/sling6/org/apache/sling/api/resource/ResourceMetadata.html > >> >> 3 http://sling.apache.org/site/discover-sling-in-15-minutes.html > >> >> 4 http://www.osgi.org/download/r4v42/r4.cmpn.pdf (chapter 112) > >> >> > >> >> > >> > http://felix.apache.org/documentation/subprojects/apache-felix-maven-scr-plugin/scr-annotations.html > >> >> > >> >> > >> > http://felix.apache.org/documentation/subprojects/apache-felix-maven-scr-plugin/extending-scr-annotations.html > >> >> > >> >> On 31 March 2013 21:58, Dishara Wijewardana <[email protected] > > > >> >> wrote: > >> >> > Hi Ian, > >> >> > Thank you very much for the explanation. Before replying to this > >> mail, I > >> >> > revisited the facts you mentioned and try to comeup with a end to > >> end big > >> >> > picture and what are the challenges that has to face when > >> implementing > >> >> this > >> >> > project. > >> >> > > >> >> > Read/write with Cassandra data seems is pretty straight forward > with > >> few > >> >> > lines of code using a client API. But the tricky part is to make a > >> clean > >> >> > bridge between JCR wrapped sling resource API vs Cassendra column > >> family > >> >> > data storage. So I took some time and went through the JCR spec and > >> try > >> >> to > >> >> > understand how it deals with resources (I assumed sling resource is > >> >> > directly based on JCR node concept). And got a good understanding > of > >> how > >> >> > JCR thinks on resource and how they deal with it. > >> >> > > >> >> > Because we need to think of the mapping between the sling wrapper > >> >> interface > >> >> > for resources which is org.apache.sling.api.resource.Resource > >> (which is > >> >> a > >> >> > JCR Node as I understand) and the Cassendra data layer. For > instance > >> >> > Cassandra provider will return a sling resource and it should be > >> enrich > >> >> > with the properties/attributes which helps the sling resource to > >> keep its > >> >> > state like resource meta data, resource type (which should be the > JCR > >> >> node > >> >> > type), and etc. > >> >> > > >> >> > And the provider should only return a resource which only has such > >> very > >> >> > basic meta data. For instance, like > >> >> org.apache.sling.api.resource.Resource > >> >> > #getChild() #getChildren() we should not keep those in memory. We > >> should > >> >> > return them on the fly from Cassandra. > >> >> > > >> >> > I think we should write a separate Sling Cassandra Adapter layer > and > >> >> > provider should talk to Cassandra through Cassandra Adapter. I hope > >> this > >> >> > will make it more cleaner. > >> >> > Appreciate your valuable feedback. So that based on feedback I can > >> >> provide > >> >> > a patch which will reflect the basic architecture and keep on > >> patching > >> >> with > >> >> > future additions. > >> >> > > >> >> > > >> >> > On Fri, Mar 29, 2013 at 3:48 AM, Ian Boston <[email protected]> wrote: > >> >> > > >> >> >> Hi and welcome, > >> >> >> Some comments inline below. > >> >> >> > >> >> >> On 29 March 2013 06:02, Dishara Wijewardana < > >> [email protected]> > >> >> >> wrote: > >> >> >> > Hi all, > >> >> >> > I am Dishara Wijewardana, a student who is willing to take part > in > >> >> this > >> >> >> > GSoC 2013 . > >> >> >> > > >> >> >> > I have successfully completed GSoC 2012 in Apache Velocity and > >> there I > >> >> >> have > >> >> >> > implemented JSR 223 support for Velocity. I found myself really > >> >> >> interested > >> >> >> > in this project since it covers very useful and interesting > >> topics. So > >> >> >> > thought of getting in to this project idea and provide a good > >> proposal > >> >> >> for > >> >> >> > this project. > >> >> >> > > >> >> >> > So I did some research around sling which might be useful for me > >> to > >> >> get > >> >> >> in > >> >> >> > to this project. I like sling as it sticks to community > standards > >> >> where > >> >> >> it > >> >> >> > uses a standard JCR2 repository to store resources which is a > >> really > >> >> good > >> >> >> > thing to have. > >> >> >> > > >> >> >> > I went through the information provided in the JIRA[1] and > >> according > >> >> to > >> >> >> > that at the end of this project what is expected to have > >> implemented > >> >> is a > >> >> >> > ResourceProvider for Sling which tunnels with a Cassandra > >> (standalone > >> >> >> > one/cluster). > >> >> >> > >> >> >> yes, correct. > >> >> >> > >> >> >> > > >> >> >> > As far as I got to know, sling directly calls to Apache > JackRabbit > >> >> APIs > >> >> >> > (JCR APIs) to store resources. So I found a bit complicated this > >> >> project > >> >> >> > idea in that sense. Because if we are to implement a Cassandra > >> backend > >> >> >> for > >> >> >> > Sling (as per this proposal), and Sling storage is on top of > >> >> JackRabbit, > >> >> >> > ideally what should happen is to make JackRabbit capable of > using > >> >> >> Cassandra > >> >> >> > as its resource persistent layer, and configure it through > Sling ? > >> >> Please > >> >> >> > correct me If I am wrong. > >> >> >> > >> >> >> Your right. > >> >> >> The idea is this, Sling resolves paths into Resources > >> >> >> ie /content/mywebsite/page1.html is resolved to a Resource with a > >> path > >> >> >> of /content/mywebsite/page1 See [1]. Normally a JCR repository > >> >> >> takes ownership of everything under /, so all Resources are JCR > >> >> >> Resources. > >> >> >> > >> >> >> However, with a ResourceProvider its possible to "mount" a > >> alternative > >> >> >> source of Resources at any location in the tree. eg: > >> >> >> If I create a ResourceProvider and configure it to respond to all > >> >> >> resource resolution operations at > >> >> >> /content/cassandra > >> >> >> > >> >> >> then > >> >> >> /content/cassandra/columnFamilyA/cassandraRowIDB > >> >> >> > >> >> >> will generate a Cassandra Resource instead of a JCR Resource. > >> >> >> > >> >> >> Initially the aim is to write a ResourceProvider that will allow > >> >> >> Readonly access to a Cassandra cluster (cluster of one is ok for > >> >> >> testing), but ultimately we would like to be able to write to that > >> >> >> cluster as well. > >> >> >> > >> >> >> Why Do it ? > >> >> >> Every storage platform has different characteristics, some are > ideal > >> >> >> for extreem volume writes of throw away data, some are ideal for > >> >> >> extreem volume reads of precious audited transactrional data. > Being > >> >> >> able to "mount" multiple stores in Sling enables Sling to > integrate > >> >> >> data from all types of sources using best of breed address each > use > >> >> >> case. (Thats the theory, anyway :)) > >> >> >> > >> >> > > >> >> > +1 and this is a wonderful architecture interms of extensibility. > >> >> Something > >> >> > even a repository vendor like Jackrabbit also would want to follow. > >> >> Because > >> >> > they only have a JCR interfaced tree. > >> >> > > >> >> > > >> >> >> > >> >> >> I hope that makes things clearer. > >> >> >> > >> >> >> 1 http://sling.apache.org/site/resources.html > >> >> >> > >> >> >> > > >> >> >> > But if it is only to READ resources, this project is relatively > >> less > >> >> >> > complex (not quite sure though ;-) ) since what is required is > to > >> >> have a > >> >> >> > JCR/Sling Resource compatible wrapper layer interface on top of > >> >> Cassendra > >> >> >> > to read cassandra data. > >> >> >> > >> >> >> Initially, just read. Then read with access control. The > read/write > >> >> >> with access control. > >> >> >> > >> >> > > >> >> > Read/Write complexity will be more or less the same as I feel. But > >> read > >> >> > write with access control is something we have to discuss > separately. > >> >> > Does sling maintaining access control directly with jackrabbit's > >> >> > javax.jcr.security module ? Or any inhouse access control layer ? > >> >> > > >> >> > > >> >> > > >> >> >> > >> >> >> > > >> >> >> > Appreciate any feedback and guidance on how to proceed. > >> >> >> > >> >> >> If you havent already you need to checkout the information at: > >> >> >> * http://www.google-melange.com/gsoc/homepage/google/gsoc2013 > >> >> >> * http://community.apache.org/gsoc.html > >> >> >> > >> >> >> especially the timeline and dates. > >> >> >> > >> >> >> There is no guarantee that Apache will be a GSoC organisation > >> >> >> (although its highly likely), and there are currently 129 project > >> >> >> proposals so there is no guarantee that you will get accepted as a > >> >> >> Student on this project, but the quality of your submission and > your > >> >> >> enthusiasm will go a long way to making that happen. > >> >> >> > >> >> >> Good luck and I look forward to seeing you on these lists over the > >> >> >> summer. If you do make it through, I and everyone in this > community > >> >> >> will try and make it fun and rewarding for you. > >> >> >> > >> >> >> Best Regards > >> >> >> Ian > >> >> >> > >> >> >> > >> >> >> > >> >> >> > > >> >> >> > [1] - https://issues.apache.org/jira/browse/SLING-2798 > >> >> >> > > >> >> >> > -- > >> >> >> > Thanks > >> >> >> > /Dishara > >> >> >> > >> >> > > >> >> > > >> >> > > >> >> > -- > >> >> > Thanks > >> >> > /Dishara > >> >> > >> > > >> > > >> > > >> > -- > >> > Thanks > >> > /Dishara > >> > > > > > > > > -- > > Thanks > > /Dishara > > > > > > -- > Thanks > /Dishara >
