Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> I think the discussion about the adapter pattern is orthogonal to the binary For me its tied to how you are going to implement this support. Adaptable patterns is one way based on my current understand of Oak design. At level of ValueFactory.getBinary we do not know if the Blob can provide a signed url. Its deep down in the layer JCR Binary -> Oak Blob -> DataStoreBlob -> S3DataStore DataRecord. So each Blob cannot provided a signed url and it depends on backing DataStore. This can be easily supported via adaptor pattern where JCR layer tries to adapt and then final backing BlobStore impl decides to provide the adaption implementation. I do not see how instance of checks can be expressed across all these layers A rough sketch of any alternative proposal would be helpful to decide how to move forward Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 09:06, Chetan Mehrotra wrote: I think the discussion about the adapter pattern is orthogonal to the binary For me its tied to how you are going to implement this support. Adaptable patterns is one way based on my current understand of Oak design. At level of ValueFactory.getBinary we do not know if the Blob can provide a signed url. Its deep down in the layer JCR Binary -> Oak Blob -> DataStoreBlob -> S3DataStore DataRecord. So each Blob cannot provided a signed url and it depends on backing DataStore. This can be easily supported via adaptor pattern where JCR layer tries to adapt and then final backing BlobStore impl decides to provide the adaption implementation. I do not see how instance of checks can be expressed across all these layers Fair point. So this is more about dynamic adaptability than future extendibility. But AFIU this could still be achieved without the full adaptable machinery: if (binProp instanceOf SignableBin) { URI uri = ((SignableBin) binProp).getSignedURI(); if (uri != null) { // resolve URI etc. } } Or alternatively something along the lines of: URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); Michael A rough sketch of any alternative proposal would be helpful to decide how to move forward Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> Fair point. So this is more about dynamic adaptability than future > extendibility. But AFIU this could still be achieved without the full > adaptable machinery: > > if (binProp instanceOf SignableBin) { > URI uri = ((SignableBin) binProp).getSignedURI(); > if (uri != null) { > // resolve URI etc. > } > } This would be tricky. The current logic is like below. 1. Oak JCR BinaryImpl holds a ValueImpl 2. ValueImpl -> PropertyState -> Blob 3. From Blob following paths are possible - Blob -> SegmentBlob -> BlobStoreBlob -> DataRecord -> S3DataRecord - Blob -> ArrayBasedBlob - Blob ... MongoBlob So at JCR level where we have a PropertyState we cannot determine if the Blob provided by it can provide a signed binary without adding such instance of check at each place. Hence the adaptor based proposal Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24 August 2017 at 08:18, Michael Dürig wrote: > > > URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); > > +1 One point Users in Sling dont know abou Oak, they know about JCR. URI uri = ((OakValueFactory) valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); No new APIs, let OakValueFactory work it out and return null if it cant do it. It should also handle a null parameter. (I assume OakValueFactory already exists) If you want to make it extensible T convertTo(Object source, Class target); used as URI uri = ((OakValueFactory) valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); The user doesnt know or need to know the URI is signed, it needs a URI that can be resolved. Oak wants it to be signed. Best Regards Ian > Michael > > > > >> A rough sketch of any alternative proposal would be helpful to decide >> how to move forward >> >> Chetan Mehrotra >> >>
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
The datastore should understand how to go from Blob -> URI. In the case of S3 it does and uses Blob.getContentId(). If the datastore doesnt know how to do it, then its not supported by the datastore. You might need a DataStore.getSignedURI(Blob b) method. On 24 August 2017 at 08:27, Chetan Mehrotra wrote: > > Fair point. So this is more about dynamic adaptability than future > > extendibility. But AFIU this could still be achieved without the full > > adaptable machinery: > > > > if (binProp instanceOf SignableBin) { > > URI uri = ((SignableBin) binProp).getSignedURI(); > > if (uri != null) { > > // resolve URI etc. > > } > > } > > This would be tricky. The current logic is like below. > > 1. Oak JCR BinaryImpl holds a ValueImpl > 2. ValueImpl -> PropertyState -> Blob > 3. From Blob following paths are possible >- Blob -> SegmentBlob -> BlobStoreBlob -> DataRecord -> S3DataRecord >- Blob -> ArrayBasedBlob >- Blob ... MongoBlob > > So at JCR level where we have a PropertyState we cannot determine if > the Blob provided by it can provide a signed binary without adding > such instance of check at each place. Hence the adaptor based proposal > > Chetan Mehrotra >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 09:27, Ian Boston wrote: On 24 August 2017 at 08:18, Michael Dürig wrote: URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); +1 One point Users in Sling dont know abou Oak, they know about JCR. URI uri = ((OakValueFactory) valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); No new APIs, let OakValueFactory work it out and return null if it cant do it. It should also handle a null parameter. (I assume OakValueFactory already exists) No, OakValueFactory does not exist as API (yet). But adding it would be more inline with how we approached the Oak API traditionally. I'm not against introducing the adaptable pattern but would like to understand whether there is concrete enough use cases beyond the current one to warrant it. Michael If you want to make it extensible T convertTo(Object source, Class target); used as URI uri = ((OakValueFactory) valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); The user doesnt know or need to know the URI is signed, it needs a URI that can be resolved. Oak wants it to be signed. Best Regards Ian Michael A rough sketch of any alternative proposal would be helpful to decide how to move forward Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24 August 2017 at 09:16, Michael Dürig wrote: > > > On 24.08.17 09:27, Ian Boston wrote: > >> On 24 August 2017 at 08:18, Michael Dürig wrote: >> >> >>> >>> URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); >>> >>> >>> +1 >> >> One point >> Users in Sling dont know abou Oak, they know about JCR. >> >> URI uri = ((OakValueFactory) >> valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); >> >> No new APIs, let OakValueFactory work it out and return null if it cant do >> it. It should also handle a null parameter. >> (I assume OakValueFactory already exists) >> > > No, OakValueFactory does not exist as API (yet). But adding it would be > more inline with how we approached the Oak API traditionally. > > If it doesnt exist then perhaps Oak could add a Service interface that deals with conversions, rather than expose a second adaptable pattern in Sling, or require type casting and instanceof. How Oak implements those conversions is upto Oak. eg public interface OakConversionService { T convertTo(Object source, Class target); } implemented as an OSGi Service used as eg @Reference private OakConversionService conversionService; public void doFilter(...) { ... URL u = conversionService.convertTo(jcrBinary, URI.class); } Best Regards Ian > I'm not against introducing the adaptable pattern but would like to > understand whether there is concrete enough use cases beyond the current > one to warrant it. > > Michael > > > >> If you want to make it extensible >> >> T convertTo(Object source, Class target); >> >> used as >> >> URI uri = ((OakValueFactory) >> valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); >> >> The user doesnt know or need to know the URI is signed, it needs a URI >> that >> can be resolved. >> Oak wants it to be signed. >> >> Best Regards >> Ian >> >> >> >> Michael >>> >>> >>> >>> >>> A rough sketch of any alternative proposal would be helpful to decide how to move forward Chetan Mehrotra >>
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
As explained in previous mail adaptable pattern requirement is to enable such a support within Oak itself. due to multiple layers involved. > If it doesnt exist then perhaps Oak could add a Service interface that > deals with conversions, rather than expose a second adaptable pattern in > Sling, or require type casting and instanceof. We can expose such a service also if that helps. That service implementation internally would anyway have to use adaptable pattern. public class OakConversionService{ AdapterType adaptTo(Binary b, Class type) { if (b instanceof Adaptable) { return (type)b.adaptTo(type); } return null } } So just another level of abstraction. Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
Il giorno gio 24 ago 2017 alle ore 10:16 Michael Dürig ha scritto: > > > On 24.08.17 09:27, Ian Boston wrote: > > On 24 August 2017 at 08:18, Michael Dürig wrote: > > > >> > >> > >> URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); > >> > >> > > +1 > > > > One point > > Users in Sling dont know abou Oak, they know about JCR. > > > > URI uri = ((OakValueFactory) > > valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); > > > > No new APIs, let OakValueFactory work it out and return null if it cant > do > > it. It should also handle a null parameter. > > (I assume OakValueFactory already exists) > > No, OakValueFactory does not exist as API (yet). But adding it would be > more inline with how we approached the Oak API traditionally. > > I'm not against introducing the adaptable pattern but would like to > understand whether there is concrete enough use cases beyond the current > one to warrant it. > +1, currently I much prefer such a concrete approach over the adaptable; not that it's bad per se, just I am not sure there'll be other use cases. That said, although I understand the use cases and requirements, this still seems to me a break into the fundamental Oak design, sorry. It's not I'm totally against it, it's just that perhaps we should rethink some of our design if it can't stick to our requirements, e.g. provide an API for accessing binaries at the storage layer, which we expect to be used by consumers having the right privileges by decorating it with something similar to a JWT [1] token, emitted by the repo. my 2 cents, Tommaso [1] : https://jwt.io/ > > Michael > > > > > If you want to make it extensible > > > > T convertTo(Object source, Class target); > > > > used as > > > > URI uri = ((OakValueFactory) > > valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); > > > > The user doesnt know or need to know the URI is signed, it needs a URI > that > > can be resolved. > > Oak wants it to be signed. > > > > Best Regards > > Ian > > > > > > > >> Michael > >> > >> > >> > >> > >>> A rough sketch of any alternative proposal would be helpful to decide > >>> how to move forward > >>> > >>> Chetan Mehrotra > >>> > >>> > > >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
I understand the difficulties involved with implementing this due to the various layers involved. The bit I don't understand is how the adaptable pattern would make those go away. To me that pattern is just another way to implement this but it would also need to deal with all those layers. Michael On 24.08.17 11:08, Chetan Mehrotra wrote: As explained in previous mail adaptable pattern requirement is to enable such a support within Oak itself. due to multiple layers involved. If it doesnt exist then perhaps Oak could add a Service interface that deals with conversions, rather than expose a second adaptable pattern in Sling, or require type casting and instanceof. We can expose such a service also if that helps. That service implementation internally would anyway have to use adaptable pattern. public class OakConversionService{ AdapterType adaptTo(Binary b, Class type) { if (b instanceof Adaptable) { return (type)b.adaptTo(type); } return null } } So just another level of abstraction. Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
Hi On Thu, Aug 24, 2017 at 9:27 AM, Ian Boston wrote: > On 24 August 2017 at 08:18, Michael Dürig wrote: > >> >> >> URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); >> >> > +1 > > One point > Users in Sling dont know abou Oak, they know about JCR. I think this issue should be solved in two steps: 1. Figure out how to surface a signed URL from the DataStore to the level of the JCR (or Oak) API. 2. Provide OSGi glue inside Sling, possibly exposing the signed URL it via adaptTo(). > > URI uri = ((OakValueFactory) > valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); > > No new APIs, let OakValueFactory work it out and return null if it cant do > it. It should also handle a null parameter. > (I assume OakValueFactory already exists) > > If you want to make it extensible > > T convertTo(Object source, Class target); > > used as > > URI uri = ((OakValueFactory) > valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); There is an upcoming OSGi Spec for a Converter service (RFC 215 Object Conversion, also usable outside of OSGI)[0]. It has an implementation in Felix, but afaik no releases so far. A generic Converter would certainly help with decoupling. Basically the S3-DataStore could register an appropriate conversion, hiding all implementation details. Regards Julian [0] https://github.com/osgi/design/blob/05cd5cf03d4b6f8a512886eae472a6b6fde594b0/rfcs/rfc0215/rfc-0215-object-conversion.pdf > > The user doesnt know or need to know the URI is signed, it needs a URI that > can be resolved. > Oak wants it to be signed. > > Best Regards > Ian > > > >> Michael >> >> >> >> >>> A rough sketch of any alternative proposal would be helpful to decide >>> how to move forward >>> >>> Chetan Mehrotra >>> >>>
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
Hi, On 24 August 2017 at 10:20, Julian Sedding wrote: > Hi > > On Thu, Aug 24, 2017 at 9:27 AM, Ian Boston wrote: > > On 24 August 2017 at 08:18, Michael Dürig wrote: > > > >> > >> > >> URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); > >> > >> > > +1 > > > > One point > > Users in Sling dont know abou Oak, they know about JCR. > > I think this issue should be solved in two steps: > > 1. Figure out how to surface a signed URL from the DataStore to the > level of the JCR (or Oak) API. > 2. Provide OSGi glue inside Sling, possibly exposing the signed URL it > via adaptTo(). > > > > > URI uri = ((OakValueFactory) > > valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); > > > > No new APIs, let OakValueFactory work it out and return null if it cant > do > > it. It should also handle a null parameter. > > (I assume OakValueFactory already exists) > > > > If you want to make it extensible > > > > T convertTo(Object source, Class target); > > > > used as > > > > URI uri = ((OakValueFactory) > > valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); > > There is an upcoming OSGi Spec for a Converter service (RFC 215 Object > Conversion, also usable outside of OSGI)[0]. It has an implementation > in Felix, but afaik no releases so far. > > A generic Converter would certainly help with decoupling. Basically > the S3-DataStore could register an appropriate conversion, hiding all > implementation details. > Sounds like a good fit. +1 Best Regards Ian > > Regards > Julian > > [0] https://github.com/osgi/design/blob/05cd5cf03d4b6f8a512886eae472a6 > b6fde594b0/rfcs/rfc0215/rfc-0215-object-conversion.pdf > > > > > The user doesnt know or need to know the URI is signed, it needs a URI > that > > can be resolved. > > Oak wants it to be signed. > > > > Best Regards > > Ian > > > > > > > >> Michael > >> > >> > >> > >> > >>> A rough sketch of any alternative proposal would be helpful to decide > >>> how to move forward > >>> > >>> Chetan Mehrotra > >>> > >>> >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> various layers involved. The bit I don't understand is how the adaptable > pattern would make those go away. To me that pattern is just another way to > implement this but it would also need to deal with all those layers. Yes this adapter support would need to be implement at all layers. So call to 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. Blob interface would extend adaptable 3. results in SegmentBlob delegating to BlobStoreBlob which 4. delegates to BlobStore // Here just passing the BlobId 5. which delegates to DataStoreBlobStore 6. which delegates to S3DataStore 7. which returns the SignedBinary implementation However adapter support would allow us to make this instance of check extensible. Otherwise we would be hardcoding instance of check to SignedBinary at each of the above place though those layers need not be aware of SignedBinary support (its specific to S3 impl) Chetan Mehrotra
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> 1. Figure out how to surface a signed URL from the DataStore to the > level of the JCR (or Oak) API. > 2. Provide OSGi glue inside Sling, possibly exposing the signed URL it > via adaptTo(). Thats sums up the requirement well. Most of proposal here is for #1. Once we have that implemented #2 can be done in Sling side Chetan Mehrotra On Thu, Aug 24, 2017 at 3:06 AM, Ian Boston wrote: > Hi, > > On 24 August 2017 at 10:20, Julian Sedding wrote: > >> Hi >> >> On Thu, Aug 24, 2017 at 9:27 AM, Ian Boston wrote: >> > On 24 August 2017 at 08:18, Michael Dürig wrote: >> > >> >> >> >> >> >> URI uri = ((OakValueFactory) valueFactory).getSignedURI(binProp); >> >> >> >> >> > +1 >> > >> > One point >> > Users in Sling dont know abou Oak, they know about JCR. >> >> I think this issue should be solved in two steps: >> >> 1. Figure out how to surface a signed URL from the DataStore to the >> level of the JCR (or Oak) API. >> 2. Provide OSGi glue inside Sling, possibly exposing the signed URL it >> via adaptTo(). >> >> > >> > URI uri = ((OakValueFactory) >> > valueFactory).getSignedURI(jcrNode.getProperty("jcr:data")); >> > >> > No new APIs, let OakValueFactory work it out and return null if it cant >> do >> > it. It should also handle a null parameter. >> > (I assume OakValueFactory already exists) >> > >> > If you want to make it extensible >> > >> > T convertTo(Object source, Class target); >> > >> > used as >> > >> > URI uri = ((OakValueFactory) >> > valueFactory). convertTo(jcrNode.getProperty("jcr:data"), URI.class); >> >> There is an upcoming OSGi Spec for a Converter service (RFC 215 Object >> Conversion, also usable outside of OSGI)[0]. It has an implementation >> in Felix, but afaik no releases so far. >> >> A generic Converter would certainly help with decoupling. Basically >> the S3-DataStore could register an appropriate conversion, hiding all >> implementation details. >> > > Sounds like a good fit. > +1 > > Best Regards > Ian > > >> >> Regards >> Julian >> >> [0] https://github.com/osgi/design/blob/05cd5cf03d4b6f8a512886eae472a6 >> b6fde594b0/rfcs/rfc0215/rfc-0215-object-conversion.pdf >> >> > >> > The user doesnt know or need to know the URI is signed, it needs a URI >> that >> > can be resolved. >> > Oak wants it to be signed. >> > >> > Best Regards >> > Ian >> > >> > >> > >> >> Michael >> >> >> >> >> >> >> >> >> >>> A rough sketch of any alternative proposal would be helpful to decide >> >>> how to move forward >> >>> >> >>> Chetan Mehrotra >> >>> >> >>> >>
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
Hi, That would require javax.jcr.Binary to implement Adaptable, which it cant. (OakBinary could but it doesnt need to). Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with Oaks internal version of the same) What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary provided by the S3DataStore itself. Since javax.jcr.Binary cant extend Adaptable, its not possible to call binary.adaptTo(SignedBinary.class) without a cast, hence, the call is done via the AdapterManager[1] SignedBinary signedBinary = adapterManager.getAdapter(binary, SignedBinary.class); --- You could just jump to URI uri = adapterManager.getAdapter(binary, URI.class); No API changes to any existing Oak APIs, Best Regards Ian 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/adapter/AdapterManager.html On 24 August 2017 at 12:38, Chetan Mehrotra wrote: > > various layers involved. The bit I don't understand is how the adaptable > > pattern would make those go away. To me that pattern is just another way > to > > implement this but it would also need to deal with all those layers. > > Yes this adapter support would need to be implement at all layers. > > So call to > 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary > 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. > Blob interface would extend adaptable > 3. results in SegmentBlob delegating to BlobStoreBlob which > 4. delegates to BlobStore // Here just passing the BlobId > 5. which delegates to DataStoreBlobStore > 6. which delegates to S3DataStore > 7. which returns the SignedBinary implementation > > However adapter support would allow us to make this instance of check > extensible. Otherwise we would be hardcoding instance of check to > SignedBinary at each of the above place though those layers need not > be aware of SignedBinary support (its specific to S3 impl) > > Chetan Mehrotra >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
>From the javadoc in [1] "The adaptable object may be any non-null object and is not required to implement the Adaptable interface." On 24 August 2017 at 12:54, Ian Boston wrote: > Hi, > That would require javax.jcr.Binary to implement Adaptable, which it cant. > (OakBinary could but it doesnt need to). > > Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with Oaks > internal version of the same) > > What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary > provided by the S3DataStore itself. > > Since javax.jcr.Binary cant extend Adaptable, its not possible to call > binary.adaptTo(SignedBinary.class) without a cast, hence, > the call is done via the AdapterManager[1] > > SignedBinary signedBinary = adapterManager.getAdapter(binary, > SignedBinary.class); > > --- > You could just jump to > URI uri = adapterManager.getAdapter(binary, URI.class); > > No API changes to any existing Oak APIs, > > Best Regards > Ian > > > 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/adapter/ > AdapterManager.html > > > > On 24 August 2017 at 12:38, Chetan Mehrotra > wrote: > >> > various layers involved. The bit I don't understand is how the adaptable >> > pattern would make those go away. To me that pattern is just another >> way to >> > implement this but it would also need to deal with all those layers. >> >> Yes this adapter support would need to be implement at all layers. >> >> So call to >> 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary >> 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. >> Blob interface would extend adaptable >> 3. results in SegmentBlob delegating to BlobStoreBlob which >> 4. delegates to BlobStore // Here just passing the BlobId >> 5. which delegates to DataStoreBlobStore >> 6. which delegates to S3DataStore >> 7. which returns the SignedBinary implementation >> >> However adapter support would allow us to make this instance of check >> extensible. Otherwise we would be hardcoding instance of check to >> SignedBinary at each of the above place though those layers need not >> be aware of SignedBinary support (its specific to S3 impl) >> >> Chetan Mehrotra >> > >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> No API changes to any existing Oak APIs, Some API needs to be exposed. Note again Oak does not depend on Sling API. Any such integration code is implemented in Sling Base module [1]. But that module would still require some API in Oak to provide such an adaptor The adaptor proposal here is for enabling layers within Oak to allow conversion of JCR Binary instance to SignedBinary. Now how this is exposed to end user depends on usage context Outside Sling -- Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it import org.apache.jackrabbit.oak.api.Adaptable Binary b = ... SignedBinary sb = null if (b instanceof Adaptable) { sb = ((Adaptable)b).adaptTo(SignedBinary.class); } Within Sling Have an AdapterManager implemented in Sling JCR Base [1] which uses above approach Chetan Mehrotra [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston wrote: > From the javadoc in [1] > > "The adaptable object may be any non-null object and is not required to > implement the Adaptable interface." > > > On 24 August 2017 at 12:54, Ian Boston wrote: > >> Hi, >> That would require javax.jcr.Binary to implement Adaptable, which it cant. >> (OakBinary could but it doesnt need to). >> >> Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with Oaks >> internal version of the same) >> >> What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary >> provided by the S3DataStore itself. >> >> Since javax.jcr.Binary cant extend Adaptable, its not possible to call >> binary.adaptTo(SignedBinary.class) without a cast, hence, >> the call is done via the AdapterManager[1] >> >> SignedBinary signedBinary = adapterManager.getAdapter(binary, >> SignedBinary.class); >> >> --- >> You could just jump to >> URI uri = adapterManager.getAdapter(binary, URI.class); >> >> No API changes to any existing Oak APIs, >> >> Best Regards >> Ian >> >> >> 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/adapter/ >> AdapterManager.html >> >> >> >> On 24 August 2017 at 12:38, Chetan Mehrotra >> wrote: >> >>> > various layers involved. The bit I don't understand is how the adaptable >>> > pattern would make those go away. To me that pattern is just another >>> way to >>> > implement this but it would also need to deal with all those layers. >>> >>> Yes this adapter support would need to be implement at all layers. >>> >>> So call to >>> 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary >>> 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. >>> Blob interface would extend adaptable >>> 3. results in SegmentBlob delegating to BlobStoreBlob which >>> 4. delegates to BlobStore // Here just passing the BlobId >>> 5. which delegates to DataStoreBlobStore >>> 6. which delegates to S3DataStore >>> 7. which returns the SignedBinary implementation >>> >>> However adapter support would allow us to make this instance of check >>> extensible. Otherwise we would be hardcoding instance of check to >>> SignedBinary at each of the above place though those layers need not >>> be aware of SignedBinary support (its specific to S3 impl) >>> >>> Chetan Mehrotra >>> >> >>
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 13:54, Ian Boston wrote: You could just jump to URI uri = adapterManager.getAdapter(binary, URI.class); No API changes to any existing Oak APIs, +1, I think this is what we should aim for. Michael
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 13:38, Chetan Mehrotra wrote: various layers involved. The bit I don't understand is how the adaptable pattern would make those go away. To me that pattern is just another way to implement this but it would also need to deal with all those layers. Yes this adapter support would need to be implement at all layers. So call to 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. Blob interface would extend adaptable Why not just add a method Blob.getSignedURI()? This would be inline with getReference() and what we have done with ReferenceBinary. Michael 3. results in SegmentBlob delegating to BlobStoreBlob which 4. delegates to BlobStore // Here just passing the BlobId 5. which delegates to DataStoreBlobStore 6. which delegates to S3DataStore 7. which returns the SignedBinary implementation However adapter support would allow us to make this instance of check extensible. Otherwise we would be hardcoding instance of check to SignedBinary at each of the above place though those layers need not be aware of SignedBinary support (its specific to S3 impl)
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> Why not just add a method Blob.getSignedURI()? This would be inline with > getReference() and what we have done with ReferenceBinary. Can be done. But later if we decide to support adapting to say FileChannel [1] then would we be adding that to Blob. Though it may not be related to different Blob types. Having adaptable support allows to extend this later with minimal changes. Chetan Mehrotra [1] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase#UC4 On Thu, Aug 24, 2017 at 5:25 AM, Michael Dürig wrote: > > > On 24.08.17 13:38, Chetan Mehrotra wrote: >>> >>> various layers involved. The bit I don't understand is how the adaptable >>> pattern would make those go away. To me that pattern is just another way >>> to >>> implement this but it would also need to deal with all those layers. >> >> >> Yes this adapter support would need to be implement at all layers. >> >> So call to >> 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary >> 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. >> Blob interface would extend adaptable > > > Why not just add a method Blob.getSignedURI()? This would be inline with > getReference() and what we have done with ReferenceBinary. > > Michael > > >> 3. results in SegmentBlob delegating to BlobStoreBlob which >> 4. delegates to BlobStore // Here just passing the BlobId >> 5. which delegates to DataStoreBlobStore >> 6. which delegates to S3DataStore >> 7. which returns the SignedBinary implementation >> >> However adapter support would allow us to make this instance of check >> extensible. Otherwise we would be hardcoding instance of check to >> SignedBinary at each of the above place though those layers need not >> be aware of SignedBinary support (its specific to S3 impl) > > >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 14:32, Chetan Mehrotra wrote: Why not just add a method Blob.getSignedURI()? This would be inline with getReference() and what we have done with ReferenceBinary. Can be done. But later if we decide to support adapting to say FileChannel [1] then would we be adding that to Blob. Though it may not be related to different Blob types. Having adaptable support allows to extend this later with minimal changes. Which circles back to my initial concern: "According to YAGNI we should stick with instance of checks unless we already have a somewhat clear picture of future extensions." Michael Chetan Mehrotra [1] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase#UC4 On Thu, Aug 24, 2017 at 5:25 AM, Michael Dürig wrote: On 24.08.17 13:38, Chetan Mehrotra wrote: various layers involved. The bit I don't understand is how the adaptable pattern would make those go away. To me that pattern is just another way to implement this but it would also need to deal with all those layers. Yes this adapter support would need to be implement at all layers. So call to 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. Blob interface would extend adaptable Why not just add a method Blob.getSignedURI()? This would be inline with getReference() and what we have done with ReferenceBinary. Michael 3. results in SegmentBlob delegating to BlobStoreBlob which 4. delegates to BlobStore // Here just passing the BlobId 5. which delegates to DataStoreBlobStore 6. which delegates to S3DataStore 7. which returns the SignedBinary implementation However adapter support would allow us to make this instance of check extensible. Otherwise we would be hardcoding instance of check to SignedBinary at each of the above place though those layers need not be aware of SignedBinary support (its specific to S3 impl)
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> Which circles back to my initial concern: "According to YAGNI we should stick > with instance of checks unless we already have a somewhat clear picture of > future extensions." I thought that with all those discussion around JCR Usecases for past some time we have an agreement for such cases (specially UC3 and UC4). Hence the push for this approach to enable further work on them going forward. Chetan Mehrotra On Thu, Aug 24, 2017 at 5:41 AM, Michael Dürig wrote: > > > On 24.08.17 14:32, Chetan Mehrotra wrote: >>> >>> Why not just add a method Blob.getSignedURI()? This would be inline with >>> getReference() and what we have done with ReferenceBinary. >> >> >> Can be done. But later if we decide to support adapting to say >> FileChannel [1] then would we be adding that to Blob. Though it may >> not be related to different Blob types. >> >> Having adaptable support allows to extend this later with minimal changes. > > > Which circles back to my initial concern: "According to YAGNI we should > stick with instance of checks unless we already have a somewhat clear > picture of future extensions." > > Michael > > >> >> Chetan Mehrotra >> [1] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase#UC4 >> >> >> On Thu, Aug 24, 2017 at 5:25 AM, Michael Dürig wrote: >>> >>> >>> >>> On 24.08.17 13:38, Chetan Mehrotra wrote: > > > various layers involved. The bit I don't understand is how the > adaptable > pattern would make those go away. To me that pattern is just another > way > to > implement this but it would also need to deal with all those layers. Yes this adapter support would need to be implement at all layers. So call to 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. Blob interface would extend adaptable >>> >>> >>> >>> Why not just add a method Blob.getSignedURI()? This would be inline with >>> getReference() and what we have done with ReferenceBinary. >>> >>> Michael >>> >>> 3. results in SegmentBlob delegating to BlobStoreBlob which 4. delegates to BlobStore // Here just passing the BlobId 5. which delegates to DataStoreBlobStore 6. which delegates to S3DataStore 7. which returns the SignedBinary implementation However adapter support would allow us to make this instance of check extensible. Otherwise we would be hardcoding instance of check to SignedBinary at each of the above place though those layers need not be aware of SignedBinary support (its specific to S3 impl) >>> >>> >>> >>> >
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 14:47, Chetan Mehrotra wrote: Which circles back to my initial concern: "According to YAGNI we should stick with instance of checks unless we already have a somewhat clear picture of future extensions." I thought that with all those discussion around JCR Usecases for past some time we have an agreement for such cases (specially UC3 and UC4). Hence the push for this approach to enable further work on them going forward. I wasn't referring to those use cases but to the adapter pattern. That pattern as it was proposed here is far more general than what is required for the problem at hand. It represents a shift in paradigm of what we used to do in Oak. That's why I asked for a "somewhat clear picture of future extensions". Introducing the adapter pattern in its full generality would IMO even deserve an own discussion thread instead of being piggy backed on the secure URL discussion. Michael Chetan Mehrotra On Thu, Aug 24, 2017 at 5:41 AM, Michael Dürig wrote: On 24.08.17 14:32, Chetan Mehrotra wrote: Why not just add a method Blob.getSignedURI()? This would be inline with getReference() and what we have done with ReferenceBinary. Can be done. But later if we decide to support adapting to say FileChannel [1] then would we be adding that to Blob. Though it may not be related to different Blob types. Having adaptable support allows to extend this later with minimal changes. Which circles back to my initial concern: "According to YAGNI we should stick with instance of checks unless we already have a somewhat clear picture of future extensions." Michael Chetan Mehrotra [1] https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase#UC4 On Thu, Aug 24, 2017 at 5:25 AM, Michael Dürig wrote: On 24.08.17 13:38, Chetan Mehrotra wrote: various layers involved. The bit I don't understand is how the adaptable pattern would make those go away. To me that pattern is just another way to implement this but it would also need to deal with all those layers. Yes this adapter support would need to be implement at all layers. So call to 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. Blob interface would extend adaptable Why not just add a method Blob.getSignedURI()? This would be inline with getReference() and what we have done with ReferenceBinary. Michael 3. results in SegmentBlob delegating to BlobStoreBlob which 4. delegates to BlobStore // Here just passing the BlobId 5. which delegates to DataStoreBlobStore 6. which delegates to S3DataStore 7. which returns the SignedBinary implementation However adapter support would allow us to make this instance of check extensible. Otherwise we would be hardcoding instance of check to SignedBinary at each of the above place though those layers need not be aware of SignedBinary support (its specific to S3 impl)
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
Hi, I am probably not helping as here as there are several layers and I think they are getting confused between what I am thinking and what you are thinking. I was thinking Oak exposed a service to convert along the lines of the OSCi converter service or the OakConversionService suggested earlier. Both Sling and other uses of Oak would use it. Inside Oak it would have its own version of an AdapterManager, AdapterFactory. the DataStore would implement an AdapterFactory and register it with the AdapterManager. The OakConversionService implementation would then use the AdapterManager to perform the conversion. If no AdapterFactory to adapt from JCR Binary to URI existed, then null would be returned from the OakConversionService. Thats no API changes to Blob, binary or anything. No complex transformation through multiple layers. No instanceof required and no difference between Sling and non Sling usage. It does require an Oak version of the AdapterManager and AdapterFactory concepts, but does not require anything to implement Adaptable. As I showed in the PoC, all the S3 specific implementation fits inside the S3DataStore which already does everything required to perform the conversion. It already goes from Binary -> Blob -> ContentIdentifier -> S3 Key -> S3 URL by virtue of ValueImpl.getBlob((Value)jcrBinary).getContentIdentifier() -> convert to S3key and then signed URL. If it would help, I can do a patch to show how it works. Best Regards Ian On 24 August 2017 at 13:05, Chetan Mehrotra wrote: > > No API changes to any existing Oak APIs, > > Some API needs to be exposed. Note again Oak does not depend on Sling > API. Any such integration code is implemented in Sling Base module > [1]. But that module would still require some API in Oak to provide > such an adaptor > > The adaptor proposal here is for enabling layers within Oak to allow > conversion of JCR Binary instance to SignedBinary. Now how this is > exposed to end user depends on usage context > > Outside Sling > -- > > Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it > > import org.apache.jackrabbit.oak.api.Adaptable > > Binary b = ... > SignedBinary sb = null > if (b instanceof Adaptable) { >sb = ((Adaptable)b).adaptTo(SignedBinary.class); > } > Within Sling > > > Have an AdapterManager implemented in Sling JCR Base [1] which uses > above approach > > Chetan Mehrotra > [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base > > > On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston wrote: > > From the javadoc in [1] > > > > "The adaptable object may be any non-null object and is not required to > > implement the Adaptable interface." > > > > > > On 24 August 2017 at 12:54, Ian Boston wrote: > > > >> Hi, > >> That would require javax.jcr.Binary to implement Adaptable, which it > cant. > >> (OakBinary could but it doesnt need to). > >> > >> Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with > Oaks > >> internal version of the same) > >> > >> What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary > >> provided by the S3DataStore itself. > >> > >> Since javax.jcr.Binary cant extend Adaptable, its not possible to call > >> binary.adaptTo(SignedBinary.class) without a cast, hence, > >> the call is done via the AdapterManager[1] > >> > >> SignedBinary signedBinary = adapterManager.getAdapter(binary, > >> SignedBinary.class); > >> > >> --- > >> You could just jump to > >> URI uri = adapterManager.getAdapter(binary, URI.class); > >> > >> No API changes to any existing Oak APIs, > >> > >> Best Regards > >> Ian > >> > >> > >> 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/adapter/ > >> AdapterManager.html > >> > >> > >> > >> On 24 August 2017 at 12:38, Chetan Mehrotra > >> wrote: > >> > >>> > various layers involved. The bit I don't understand is how the > adaptable > >>> > pattern would make those go away. To me that pattern is just another > >>> way to > >>> > implement this but it would also need to deal with all those layers. > >>> > >>> Yes this adapter support would need to be implement at all layers. > >>> > >>> So call to > >>> 1. binary.adaptTo(SignedBinary.class) //binary is JCR Binary > >>> 2. results in blob.adaptTo(SignedBinary.class) //blob is Oak Blob. > >>> Blob interface would extend adaptable > >>> 3. results in SegmentBlob delegating to BlobStoreBlob which > >>> 4. delegates to BlobStore // Here just passing the BlobId > >>> 5. which delegates to DataStoreBlobStore > >>> 6. which delegates to S3DataStore > >>> 7. which returns the SignedBinary implementation > >>> > >>> However adapter support would allow us to make this instance of check > >>> extensible. Otherwise we would be hardcoding instance of check to > >>> SignedBinary at each of the above place though those layers need not > >>> be aware of SignedBinary support (its specific to S3 impl) > >>> > >>> Chetan Mehrotra > >>> > >> > >> >
[ANNOUNCE] Apache Jackrabbit Oak 1.0.39 released
The Apache Jackrabbit community is pleased to announce the release of Apache Jackrabbit Oak 1.0.39. The release is available for download at: http://jackrabbit.apache.org/downloads.html See the full release notes below for details about this release: Release Notes -- Apache Jackrabbit Oak -- Version 1.0.39 Introduction Jackrabbit Oak is a scalable, high-performance hierarchical content repository designed for use as the foundation of modern world-class web sites and other demanding content applications. Apache Jackrabbit Oak 1.0.39 is a patch release that contains fixes and improvements over Oak 1.0. Jackrabbit Oak 1.0.x releases are considered stable and targeted for production use. The Oak effort is a part of the Apache Jackrabbit project. Apache Jackrabbit is a project of the Apache Software Foundation. Changes in Oak 1.0.39 - Technical task [OAK-5652] - RDB*Store: update Oracle JDBC driver reference to 12.1.0.2.0 [OAK-5667] - RDBDocumentStore: remove support for DBs without support for CASE statements in SELECT [OAK-6134] - RDB*Store: update PostgreSQL JDBC [OAK-6143] - RDB*store fixtures: shorten table name prefixes for Oracle [OAK-6226] - RDBDocumentStoreDB: missing @Override statements [OAK-6244] - RDB*Store: update postgresql JDBC driver reference to 42.1.1 [OAK-6247] - RDB*Store: update Tomcat JDBC pool dependency to 7.0.78 Bug [OAK-4148] - RAT plugin complains about derby files [OAK-4684] - RepositorySidegrade should also run commit hooks as they are run in upgrade [OAK-5579] - Test failure: ExternalPrivateStoreIT [OAK-5993] - Utils.isIdFromLongPath() may throw StringIndexOutOfBoundsException [OAK-6086] - Incorrect usage of RDBDocumentStore.unwrap() [OAK-6391] - With FastQuerySize, getSize() returns -1 if there are exactly 21 rows Improvement [OAK-4136] - release profile in maven [OAK-6351] - Invalidate cache entries when getChildNodes() is aborted Task [OAK-6132] - Backport oak-upgrade to 1.0 and 1.2 [OAK-6159] - BlobReferenceIterator: improve test coverage for RDB In addition to the above-mentioned changes, this release contains all changes included in previous Apache Jackrabbit Oak 1.0.x releases. Please note, the backported RDB support for the DocumentNodeStore is considered experimental at this point and is not yet ready for production use. Feel free to try it out and report any issues you may see to the Oak developers. For more detailed information about all the changes in this and other Oak releases, please see the Oak issue tracker at https://issues.apache.org/jira/browse/OAK Release Contents This release consists of a single source archive packaged as a zip file. The archive can be unpacked with the jar tool from your JDK installation. See the README.md file for instructions on how to build this release. The source archive is accompanied by SHA1 and MD5 checksums and a PGP signature that you can use to verify the authenticity of your download. The public key used for the PGP signature can be found athttp://www.apache.org/dist/jackrabbit/KEYS. About Apache Jackrabbit Oak --- Jackrabbit Oak is a scalable, high-performance hierarchical content repository designed for use as the foundation of modern world-class web sites and other demanding content applications. The Oak effort is a part of the Apache Jackrabbit project. Apache Jackrabbit is a project of the Apache Software Foundation. For more information, visit http://jackrabbit.apache.org/oak About The Apache Software Foundation Established in 1999, The Apache Software Foundation provides organizational, legal, and financial support for more than 140 freely-available, collaboratively-developed Open Source projects. The pragmatic Apache License enables individual and commercial users to easily deploy Apache software; the Foundation's intellectual property framework limits the legal exposure of its 3,800+ contributors. For more information, visit http://www.apache.org/
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
> Inside Oak it would have its own version of an AdapterManager, > AdapterFactory. the DataStore would implement an AdapterFactory and > register it with the AdapterManager. The OakConversionService > implementation would then use the AdapterManager to perform the conversion. > If no AdapterFactory to adapt from JCR Binary to URI existed, then null > would be returned from the OakConversionService. > > Thats no API changes to Blob, binary or anything. No complex transformation > through multiple layers. No instanceof required and no difference between > Sling and non Sling usage. > It does require an Oak version of the AdapterManager and AdapterFactory > concepts, but does not require anything to implement Adaptable. Thanks for those details. Much clear now. So with this we need not add adaptTo to all stuff. Instead provide an OakConversionService which converts the Binary to provided type and then have DataStores provide the AdapterFactory. This would indeed avoid any new methods in existing objects and provide a single entry point. +1 for this approach Chetan Mehrotra On Thu, Aug 24, 2017 at 6:16 AM, Ian Boston wrote: > Hi, > I am probably not helping as here as there are several layers and I think > they are getting confused between what I am thinking and what you are > thinking. > > I was thinking Oak exposed a service to convert along the lines of the OSCi > converter service or the OakConversionService suggested earlier. Both Sling > and other uses of Oak would use it. > > Inside Oak it would have its own version of an AdapterManager, > AdapterFactory. the DataStore would implement an AdapterFactory and > register it with the AdapterManager. The OakConversionService > implementation would then use the AdapterManager to perform the conversion. > If no AdapterFactory to adapt from JCR Binary to URI existed, then null > would be returned from the OakConversionService. > > Thats no API changes to Blob, binary or anything. No complex transformation > through multiple layers. No instanceof required and no difference between > Sling and non Sling usage. > It does require an Oak version of the AdapterManager and AdapterFactory > concepts, but does not require anything to implement Adaptable. > > As I showed in the PoC, all the S3 specific implementation fits inside the > S3DataStore which already does everything required to perform the > conversion. It already goes from Binary -> Blob -> ContentIdentifier -> S3 > Key -> S3 URL by virtue of > ValueImpl.getBlob((Value)jcrBinary).getContentIdentifier() -> convert to > S3key and then signed URL. > > If it would help, I can do a patch to show how it works. > Best Regards > Ian > > On 24 August 2017 at 13:05, Chetan Mehrotra > wrote: > >> > No API changes to any existing Oak APIs, >> >> Some API needs to be exposed. Note again Oak does not depend on Sling >> API. Any such integration code is implemented in Sling Base module >> [1]. But that module would still require some API in Oak to provide >> such an adaptor >> >> The adaptor proposal here is for enabling layers within Oak to allow >> conversion of JCR Binary instance to SignedBinary. Now how this is >> exposed to end user depends on usage context >> >> Outside Sling >> -- >> >> Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it >> >> import org.apache.jackrabbit.oak.api.Adaptable >> >> Binary b = ... >> SignedBinary sb = null >> if (b instanceof Adaptable) { >>sb = ((Adaptable)b).adaptTo(SignedBinary.class); >> } > > >> Within Sling >> >> >> Have an AdapterManager implemented in Sling JCR Base [1] which uses >> above approach >> >> Chetan Mehrotra >> [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base >> >> >> On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston wrote: >> > From the javadoc in [1] >> > >> > "The adaptable object may be any non-null object and is not required to >> > implement the Adaptable interface." >> > >> > >> > On 24 August 2017 at 12:54, Ian Boston wrote: >> > >> >> Hi, >> >> That would require javax.jcr.Binary to implement Adaptable, which it >> cant. >> >> (OakBinary could but it doesnt need to). >> >> >> >> Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with >> Oaks >> >> internal version of the same) >> >> >> >> What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary >> >> provided by the S3DataStore itself. >> >> >> >> Since javax.jcr.Binary cant extend Adaptable, its not possible to call >> >> binary.adaptTo(SignedBinary.class) without a cast, hence, >> >> the call is done via the AdapterManager[1] >> >> >> >> SignedBinary signedBinary = adapterManager.getAdapter(binary, >> >> SignedBinary.class); >> >> >> >> --- >> >> You could just jump to >> >> URI uri = adapterManager.getAdapter(binary, URI.class); >> >> >> >> No API changes to any existing Oak APIs, >> >> >> >> Best Regards >> >> Ian >> >> >> >> >> >> 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/ad
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24.08.17 15:33, Chetan Mehrotra wrote: Inside Oak it would have its own version of an AdapterManager, AdapterFactory. the DataStore would implement an AdapterFactory and register it with the AdapterManager. The OakConversionService implementation would then use the AdapterManager to perform the conversion. If no AdapterFactory to adapt from JCR Binary to URI existed, then null would be returned from the OakConversionService. Thats no API changes to Blob, binary or anything. No complex transformation through multiple layers. No instanceof required and no difference between Sling and non Sling usage. It does require an Oak version of the AdapterManager and AdapterFactory concepts, but does not require anything to implement Adaptable. Thanks for those details. Much clear now. So with this we need not add adaptTo to all stuff. Instead provide an OakConversionService which converts the Binary to provided type and then have DataStores provide the AdapterFactory. This would indeed avoid any new methods in existing objects and provide a single entry point. +1 for this approach Yay! Michael Chetan Mehrotra On Thu, Aug 24, 2017 at 6:16 AM, Ian Boston wrote: Hi, I am probably not helping as here as there are several layers and I think they are getting confused between what I am thinking and what you are thinking. I was thinking Oak exposed a service to convert along the lines of the OSCi converter service or the OakConversionService suggested earlier. Both Sling and other uses of Oak would use it. Inside Oak it would have its own version of an AdapterManager, AdapterFactory. the DataStore would implement an AdapterFactory and register it with the AdapterManager. The OakConversionService implementation would then use the AdapterManager to perform the conversion. If no AdapterFactory to adapt from JCR Binary to URI existed, then null would be returned from the OakConversionService. Thats no API changes to Blob, binary or anything. No complex transformation through multiple layers. No instanceof required and no difference between Sling and non Sling usage. It does require an Oak version of the AdapterManager and AdapterFactory concepts, but does not require anything to implement Adaptable. As I showed in the PoC, all the S3 specific implementation fits inside the S3DataStore which already does everything required to perform the conversion. It already goes from Binary -> Blob -> ContentIdentifier -> S3 Key -> S3 URL by virtue of ValueImpl.getBlob((Value)jcrBinary).getContentIdentifier() -> convert to S3key and then signed URL. If it would help, I can do a patch to show how it works. Best Regards Ian On 24 August 2017 at 13:05, Chetan Mehrotra wrote: No API changes to any existing Oak APIs, Some API needs to be exposed. Note again Oak does not depend on Sling API. Any such integration code is implemented in Sling Base module [1]. But that module would still require some API in Oak to provide such an adaptor The adaptor proposal here is for enabling layers within Oak to allow conversion of JCR Binary instance to SignedBinary. Now how this is exposed to end user depends on usage context Outside Sling -- Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it import org.apache.jackrabbit.oak.api.Adaptable Binary b = ... SignedBinary sb = null if (b instanceof Adaptable) { sb = ((Adaptable)b).adaptTo(SignedBinary.class); } Within Sling Have an AdapterManager implemented in Sling JCR Base [1] which uses above approach Chetan Mehrotra [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston wrote: From the javadoc in [1] "The adaptable object may be any non-null object and is not required to implement the Adaptable interface." On 24 August 2017 at 12:54, Ian Boston wrote: Hi, That would require javax.jcr.Binary to implement Adaptable, which it cant. (OakBinary could but it doesnt need to). Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with Oaks internal version of the same) What is needed is an AdapterFactory for javax.jcr.Binary to SignedBinary provided by the S3DataStore itself. Since javax.jcr.Binary cant extend Adaptable, its not possible to call binary.adaptTo(SignedBinary.class) without a cast, hence, the call is done via the AdapterManager[1] SignedBinary signedBinary = adapterManager.getAdapter(binary, SignedBinary.class); --- You could just jump to URI uri = adapterManager.getAdapter(binary, URI.class); No API changes to any existing Oak APIs, Best Regards Ian 1 https://sling.apache.org/apidocs/sling5/org/apache/sling/api/adapter/ AdapterManager.html On 24 August 2017 at 12:38, Chetan Mehrotra wrote: various layers involved. The bit I don't understand is how the adaptable pattern would make those go away. To me that pattern is just another way to implement this but it would also need to deal with all those layers.
Re: OAK-6575 - Provide a secure external URL to a DataStore binary.
On 24 August 2017 at 14:42, Michael Dürig wrote: > > > On 24.08.17 15:33, Chetan Mehrotra wrote: > >> Inside Oak it would have its own version of an AdapterManager, >>> AdapterFactory. the DataStore would implement an AdapterFactory and >>> register it with the AdapterManager. The OakConversionService >>> implementation would then use the AdapterManager to perform the >>> conversion. >>> If no AdapterFactory to adapt from JCR Binary to URI existed, then null >>> would be returned from the OakConversionService. >>> >>> Thats no API changes to Blob, binary or anything. No complex >>> transformation >>> through multiple layers. No instanceof required and no difference between >>> Sling and non Sling usage. >>> It does require an Oak version of the AdapterManager and AdapterFactory >>> concepts, but does not require anything to implement Adaptable. >>> >> >> Thanks for those details. Much clear now. So with this we need not add >> adaptTo to all stuff. Instead provide an OakConversionService which >> converts the Binary to provided type and then have DataStores provide >> the AdapterFactory. >> >> This would indeed avoid any new methods in existing objects and >> provide a single entry point. >> >> +1 for this approach >> > > Yay! Rough quick PoC at [1] to ilustrate how it might work. 1 https://github.com/apache/jackrabbit-oak/compare/trunk...ieb:OAK-6575?expand=1 > > > Michael > > > >> Chetan Mehrotra >> >> >> On Thu, Aug 24, 2017 at 6:16 AM, Ian Boston wrote: >> >>> Hi, >>> I am probably not helping as here as there are several layers and I think >>> they are getting confused between what I am thinking and what you are >>> thinking. >>> >>> I was thinking Oak exposed a service to convert along the lines of the >>> OSCi >>> converter service or the OakConversionService suggested earlier. Both >>> Sling >>> and other uses of Oak would use it. >>> >>> Inside Oak it would have its own version of an AdapterManager, >>> AdapterFactory. the DataStore would implement an AdapterFactory and >>> register it with the AdapterManager. The OakConversionService >>> implementation would then use the AdapterManager to perform the >>> conversion. >>> If no AdapterFactory to adapt from JCR Binary to URI existed, then null >>> would be returned from the OakConversionService. >>> >>> Thats no API changes to Blob, binary or anything. No complex >>> transformation >>> through multiple layers. No instanceof required and no difference between >>> Sling and non Sling usage. >>> It does require an Oak version of the AdapterManager and AdapterFactory >>> concepts, but does not require anything to implement Adaptable. >>> >>> As I showed in the PoC, all the S3 specific implementation fits inside >>> the >>> S3DataStore which already does everything required to perform the >>> conversion. It already goes from Binary -> Blob -> ContentIdentifier -> >>> S3 >>> Key -> S3 URL by virtue of >>> ValueImpl.getBlob((Value)jcrBinary).getContentIdentifier() -> convert to >>> S3key and then signed URL. >>> >>> If it would help, I can do a patch to show how it works. >>> Best Regards >>> Ian >>> >>> On 24 August 2017 at 13:05, Chetan Mehrotra >>> wrote: >>> >>> No API changes to any existing Oak APIs, > Some API needs to be exposed. Note again Oak does not depend on Sling API. Any such integration code is implemented in Sling Base module [1]. But that module would still require some API in Oak to provide such an adaptor The adaptor proposal here is for enabling layers within Oak to allow conversion of JCR Binary instance to SignedBinary. Now how this is exposed to end user depends on usage context Outside Sling -- Check if binary instanceof Oak Adaptable. If yes then cast it and adapt it import org.apache.jackrabbit.oak.api.Adaptable Binary b = ... SignedBinary sb = null if (b instanceof Adaptable) { sb = ((Adaptable)b).adaptTo(SignedBinary.class); } >>> >>> >>> Within Sling Have an AdapterManager implemented in Sling JCR Base [1] which uses above approach Chetan Mehrotra [1] https://github.com/apache/sling/tree/trunk/bundles/jcr/base On Thu, Aug 24, 2017 at 4:55 AM, Ian Boston wrote: > From the javadoc in [1] > > "The adaptable object may be any non-null object and is not required to > implement the Adaptable interface." > > > On 24 August 2017 at 12:54, Ian Boston wrote: > > Hi, >> That would require javax.jcr.Binary to implement Adaptable, which it >> > cant. > (OakBinary could but it doesnt need to). >> >> Using Sling AdapterFactory/AdapterManger javadoc (to be replaced with >> > Oaks > internal version of the same) >> >> What is needed is an AdapterFactory for javax.jcr.Binary to >> SignedBinary >> provided by the S3DataStor
[RESULT][VOTE] Release Apache Jackrabbit Oak 1.7.6
Hi, The vote passes as follows: +1 Andrei Dulceanu +1 Robert Munteanu +1 Michael Dürig +1 Tommaso Teofili +1 Amit Jain Thanks for voting. I'll push the release out. Regards Amit
[ANNOUNCE] Apache Jackrabbit Oak 1.7.6 released
The Apache Jackrabbit community is pleased to announce the release of Apache Jackrabbit Oak 1.7.6. The release is available for download at: http://jackrabbit.apache.org/downloads.html See the full release notes below for details about this release: Release Notes -- Apache Jackrabbit Oak -- Version 1.7.6 Introduction Jackrabbit Oak is a scalable, high-performance hierarchical content repository designed for use as the foundation of modern world-class web sites and other demanding content applications. Apache Jackrabbit Oak 1.7.6 is an unstable release cut directly from Jackrabbit Oak trunk, with a focus on new features and other improvements. For production use we recommend the latest stable 1.6.x release. The Oak effort is a part of the Apache Jackrabbit project. Apache Jackrabbit is a project of the Apache Software Foundation. Changes in Oak 1.7.6 - Sub-task [OAK-6445] - Ensure mounted node stores don't contain versionable nodes Technical task [OAK-6505] - Ensure mounted node stores don't contain referenceable nodes [OAK-6524] - Provide an extension point to customize the NodeStore builders [OAK-6546] - JsonSerializer should taken an instance of JsopWriter Bug [OAK-6372] - ListRecord cannot handle more than 16581375 entries [OAK-6416] - Test failure: MapRecordTest.testOak1104 [OAK-6452] - IllegalStateException: too much data for a segment during oak-upgrade from segment to segment-tar [OAK-6500] - NRTIndex leaks file handles due to unclosed IndexReader [OAK-6503] - Active deletion of blobs tries to delete sub-16k inlined blobs in case of seg-tar [OAK-6504] - Active deletion of blobs needs to indicate information about purged blobs to mark-sweep collector [OAK-6507] - Cleanup incorrectly removes base state created by full compaction [OAK-6527] - CompositeNodeStore permission evaluation fails for open setups [OAK-6529] - IndexLoaderV1 and IndexLoaderV2 should not rely on Buffer.array() [OAK-6541] - While importing new index property indexes are getting marked for reindex [OAK-6548] - Composite node builder/state keeps references to all the ancestor states [OAK-6567] - Fix OSGi wiring after netty update to 4.1.x Improvement [OAK-937] - Query engine index selection tweaks: shortcut and hint [OAK-5902] - Cold standby should allow syncing of blobs bigger than 2.2 GB [OAK-6407] - Refactor oak.spi.query into a separate module/bundle [OAK-6450] - Stop relying on the service.pid property in SecurityProviderRegistration [OAK-6497] - Support old Segment NodeStore setups for oak-run index tooling [OAK-6510] - Fix string representation of the old generation reclaimer [OAK-6518] - Implement transparent usage of different TAR index formats [OAK-6520] - Improve tail compactions resilience when base state cannot be determined [OAK-6523] - Tune DocumentNodeStore setup for indexing flow for index command [OAK-6525] - Index consistency checker should check all indexes even after failure [OAK-6526] - Slow queries: ability to track them via JMX [OAK-6528] - Implement transparent usage of different binary references index formats [OAK-6530] - Expose last RGC result via Supplier [OAK-6534] - Compute indexPaths from index definitions json [OAK-6536] - Periodic log message from continuous RGC [OAK-6537] - Don't encode the checksums in the TAR index tests [OAK-6539] - Decrease version export for org.apache.jackrabbit.oak.spi.security.authentication [OAK-6540] - Session.hasAccess(...) should reflect read-only status of mounts [OAK-6543] - NodeCounter: JMX description [OAK-6549] - Move gcType to SegmentGCOptions [OAK-6552] - Dump index command should fallback to dumping NodeState incase of error [OAK-6556] - gc.log should contain recordId of compacted root after offline compaction [OAK-6558] - Remove the 'parent' field from the CompositeNodeBuilder [OAK-6559] - Don't allow to place mounts under mounts [OAK-6564] - Update netty dependency to 4.1.x [OAK-6565] - GetBlobResponseEncoder should not write all chunks at once New Feature [OAK-3710] - Continuous revision GC [OAK-6457] - Increment the segment version number [OAK-6531] - Implement rolling upgrade from Oak 1.6 [OAK-6545] - Support for dumping repository content as json along with blobs via oak-run [OAK-6553] - Progress indicator for compaction Task [OAK-6399] - Re-establish full offline compaction functionality [OAK-6449] - Stop using the maven-scr-plugin [OAK-6498] - Update Oak trunk to Jackrabbit 2.15.5 [OAK-6511] - Switch to official OSGi versioning annotations [OAK-6519] - Properly handle tail compactions in deduplication caches [OAK-6522] - Implement unit tests for OnlineCompactor [OAK-6532] - Minimise usage of junit-addons [OAK-6533] - Adjust test classpath order to reduce number of error marker