Re: Sling retrieving out-of-date data
On Wed, Dec 2, 2015 at 7:10 PM, H Kwrote: > but is there a way to use the same Node Store when connecting through an Oak > application? NodeStore is an in vm manifestation. Before going further wanted to understand your usecase and flow. The code which connects using Oak is that to be run outside of Sling or you can have a customer servlet in Sling which you can invoke from outside > Furthermore, as for connecting over RMI: the reason why I abandoned that > cause was because unlike in Sling 7 Not sure on that part but may be you can try DavEx based remoting [1]. I believe that should work Chetan Mehrotra [1] http://wiki.apache.org/jackrabbit/RemoteAccess#DavEx
RE: Sling retrieving out-of-date data
Hi Chetan, [1] My use case is that I will have Sling 8 running, using its Oak repository underneath and using DocumentNodeStore (for MongoDB). I will then have a separate application running outside Sling that uses the JCR and Oak API, that has numerous REST services (which add/modify/add-children-to/... nodes in Sling's repository), which will not have its own repository but will instead use the one that is underneath Sling to make changes to. I hope to be able to make changes in one (Sling or my app outside Sling) and have the other notice these changes immediately, if possible. Yes, DavEx-based remoting works perfect. Thanks for the suggestion. I suppose my intention is to have a Sling framework with an Oak content repository somewhere and have numerous Oak applications connecting to the repository. I intend to serve multiple customers (multi-tenant) and likely will be employing cloud storage in the future. For various reasons, MongoDB is preferred over a SQL-based database. I've not thought enough about how things will scale in the future so I'm not sure if DavEx remoting has any disadvantage or not versus having numerous Node Stores, but I'll also continue to learn more about Oak and MongoDB to see how they fit in the picture. But for now, [1] is what I'm aiming for. Thanks, H > Date: Wed, 2 Dec 2015 19:59:03 +0530 > Subject: Re: Sling retrieving out-of-date data > From: chetan.mehro...@gmail.com > To: users@sling.apache.org > > On Wed, Dec 2, 2015 at 7:10 PM, H K <redflagba...@hotmail.com> wrote: > > but is there a way to use the same Node Store when connecting through an > > Oak application? > > NodeStore is an in vm manifestation. Before going further wanted to > understand your usecase and flow. The code which connects using Oak is > that to be run outside of Sling or you can have a customer servlet in > Sling which you can invoke from outside > > > Furthermore, as for connecting over RMI: the reason why I abandoned that > > cause was because unlike in Sling 7 > > Not sure on that part but may be you can try DavEx based remoting [1]. > I believe that should work > > Chetan Mehrotra > [1] http://wiki.apache.org/jackrabbit/RemoteAccess#DavEx
Re: Sling retrieving out-of-date data
On Tuesday 01 December 2015 19:22:14 H K wrote: > Hi, > > Sure. In pseudocode, here it is: > > 1. Have Sling standalone running and have application that provides REST > service createChildFolder (for example sake). Application connects to > repository by using DocumentMK, as in the link from the OP post. > > 2. Call createChildFolder("pathToParent", "childName"). > > 3. Make GET request to http://localhost:8080/pathToParent.infinity.json > > 4. Inspecting the JSON Object, expect there to be a child node in parent > node but none are there. Test failed. > > However, if I let the thread sleep for 2 seconds after step 2, child exists > in the JSON as wanted. It works without a sleep when running in an OSGi container using a NodeStore from DocumentNodeStoreService (test is based on SlingLaunchpadOakMongoIT). Non-OSGi setup with different NodeStore needs to be verified. Regards, O. > Thanks, > H > > > From: apa...@oliverlietz.de > > To: users@sling.apache.org > > Subject: Re: Sling retrieving out-of-date data > > Date: Tue, 1 Dec 2015 19:49:20 +0100 > > > > On Tuesday 01 December 2015 11:37:42 H K wrote: > > > Hi, > > > > Hi, > > > > > I'm currently using the Sling 8 standalone and it's connected to a > > > MongoDB > > > database. I've got an application which also connects to the same > > > MongoDB > > > database and writes/reads from it. The connection is done using > > > DocumentMK, > > > following the instructions at > > > https://jackrabbit.apache.org/oak/docs/construct.html > > > > > > My problem is that when I write data to the repository through my > > > application, Sling does not seem to get it fast enough, which may be > > > because it is caching or because it only periodically checks to see if > > > the > > > data it has is up-to-date or not. For instance in my unit tests that > > > test > > > some of my REST services, I'm creating a child node somewhere and then > > > doing a GET request to Sling to get back a JSON representing the updated > > > parent. However, the returned result does not have the child in it. I > > > know > > > the changes are made because if I delay for two seconds before > > > submitting > > > the GET request, all works fine. > > > > > > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) > > > and > > > connecting over RMI. > > > > > > Can someone provide some insight and/or a solution? Thanks! > > > > can you provide a minimal test case which resembles what you are doing? > > > > Regards, > > O.
Re: Sling retrieving out-of-date data
On Wednesday 02 December 2015 10:00:51 Chetan Mehrotra wrote: > On Tue, Dec 1, 2015 at 10:07 PM, H Kwrote: > > My problem is that when I write data to the repository through my > > application, Sling does not seem to get it fast enough, which may be > > because it is caching or because it only periodically checks to see if > > the data it has is up-to-date or not hi Chetan, > What you are observing here is the result of eventual consistency > model of Oak which is a MVCC based store [1]. When using > DocumentNodeStore (used for Mongo) each cluster node periodically > polls the backend for change in root node and then updates its head > revision to make those changes visible to current cluster node. So > change done by one cluster not would not be immediately *visible* on > other cluster nodes. but this is not the case for a single node, right? And I assume a non-cluster setup as H K didn't mention it. Regards, O. > > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) and > > connecting over RMI. > That was because there you were not using a cluster. You were > connecting to the JR running within Sling using RMI. So it was a > single node setup. If you use RMI now also and connect to same Sling > node then it would work as expected. > > Chetan Mehrotra > [1] http://jackrabbit.apache.org/oak/docs/architecture/nodestate.html
Re: Sling retrieving out-of-date data
On Wed, Dec 2, 2015 at 2:20 PM, Oliver Lietzwrote: > but this is not the case for a single node, right? And I assume a non-cluster > setup as H K didn't mention it. Code that HK mentioned above // REST service createChildFolder running on localhost:8040 public void createChildFolder(String parentPath, String childName) { DB database = new MongoClient("127.0.0.1", 27017).getDB("sling"); DocumentNodeStore store = new DocumentMK.Builder().setMongoDB(database).getNodeStore(); Repository repo = new Jcr(new Oak(store)).createRepository(); SimpleCredentials creds = new SimpleCredentials("admin", "admin".toCharArray()); Session session = (javax.jcr.Session) repository.login(creds); Node parent = session.getRootNode().getNode(parentPath); Node child = parent.addNode(childName, "sling:Folder"); session.save(); session.logout(); store.dispose(); } It technically creates a new Oak instances and thus you have a cluster! Had he connected to Sling using Remoting and not via Oak then it would be a single node case Chetan Mehrotra
Re: Sling retrieving out-of-date data
On Wednesday 02 December 2015 15:48:34 Chetan Mehrotra wrote: > On Wed, Dec 2, 2015 at 2:20 PM, Oliver Lietzwrote: > > but this is not the case for a single node, right? And I assume a > > non-cluster setup as H K didn't mention it. > > Code that HK mentioned above > > > // REST service createChildFolder running on localhost:8040 > public void createChildFolder(String parentPath, String childName) { > DB database = new MongoClient("127.0.0.1", > 27017).getDB("sling"); DocumentNodeStore store = new > DocumentMK.Builder().setMongoDB(database).getNodeStore(); > Repository repo = new Jcr(new Oak(store)).createRepository(); > SimpleCredentials creds = new SimpleCredentials("admin", > "admin".toCharArray()); > Session session = (javax.jcr.Session) repository.login(creds); > Node parent = session.getRootNode().getNode(parentPath); > Node child = parent.addNode(childName, "sling:Folder"); > session.save(); > session.logout(); > store.dispose(); > } > > > It technically creates a new Oak instances and thus you have a > cluster! Had he connected to Sling using Remoting and not via Oak then > it would be a single node case Thanks, Chetan. Is it really the _new Oak_ or the new NodeStore which sets up a cluster? I would guess the new NodeStore as in my test with the same NodeStore and new Oak no sleep was required to get the previously created node. The Oak documentation for clustering could be more expressive... O. > Chetan Mehrotra
Re: Sling retrieving out-of-date data
On Wed, Dec 2, 2015 at 4:13 PM, Oliver Lietzwrote: > Thanks, Chetan. Is it really the _new Oak_ or the new NodeStore which sets up > a cluster? Its new NodeStore > The Oak documentation for clustering could be more expressive... Yes docs are not upto the mark. Would try to find some time and update them Chetan Mehrotra
RE: Sling retrieving out-of-date data
Hi Chetan and Oliver, Thanks for the replies. So assuming that I don't connect over RMI, does this mean there is no way around this problem apart from delaying operations a few seconds? If multiple Node Stores are being used, then it seems reasonable that the backend is queried periodically, hence the lag, but is there a way to use the same Node Store when connecting through an Oak application? Furthermore, as for connecting over RMI: the reason why I abandoned that cause was because unlike in Sling 7, where I configured the Apache Sling JCR Repository RMI Registrar on port 1099 and saved it to expose the repository over RMI, it did not work in Sling 8. Neither do I see a Jackrabbit/Oak folder on my local machine, though the logs do state that RMI over 1099 is enabled. I've confirmed by attempting to telnet into "localhost 1099", which works when RMI is enabled in Sling 7 but not when it is in Sling 8. > Date: Wed, 2 Dec 2015 17:22:31 +0530 > Subject: Re: Sling retrieving out-of-date data > From: chetan.mehro...@gmail.com > To: users@sling.apache.org > > On Wed, Dec 2, 2015 at 4:13 PM, Oliver Lietz <apa...@oliverlietz.de> wrote: > > Thanks, Chetan. Is it really the _new Oak_ or the new NodeStore which sets > > up > > a cluster? > > Its new NodeStore > > > The Oak documentation for clustering could be more expressive... > > Yes docs are not upto the mark. Would try to find some time and update them > > > Chetan Mehrotra
RE: Sling retrieving out-of-date data
To further simplify the example, taking out my REST services out of the picture, simply running the following code in a Java Oak app: DB database = new MongoClient("127.0.0.1", 27017).getDB("sling"); DocumentNodeStore store = new DocumentMK.Builder().setMongoDB(database).getNodeStore(); Repository repo = new Jcr(new Oak(store)).createRepository(); Session session = repo.login( new SimpleCredentials("admin", "admin".toCharArray()) ); Node root = session.getRootNode(); Node child = root.addNode("myFolder", "nt:folder"); session.save(); // getResponseFromSling simply does a GET request to Sling to the node at path using HttpURLConnection and returns the resulting JSON System.out.println(getResponseFromSling("/myFolder"); This results in an error: java.io.FileNotFoundException: http://localhost:8080/myFolder.harray.infinity.json because it cannot get the inputStream from a Node that Sling doesn't think exists. However, adding a two (2) second delay (by sleeping) before doing the GET request to Sling returns the correct JSON successfully. Again, this problem does NOT exist using Jackrabbit and Sling 7 (as opposed to Oak with MongoDB and Sling 8). > From: redflagba...@hotmail.com > To: users@sling.apache.org > Subject: RE: Sling retrieving out-of-date data > Date: Tue, 1 Dec 2015 19:57:50 + > > It is an Oak application and yes, it does manipulate the documents directly > without making REST calls to Sling. Much of it taken from > http://jackrabbit.apache.org/oak/docs/construct.html , here is what the code > may look like: > > // REST service createChildFolder running on localhost:8040 > public void createChildFolder(String parentPath, String childName) { > DB database = new MongoClient("127.0.0.1", 27017).getDB("sling"); > DocumentNodeStore store = new > DocumentMK.Builder().setMongoDB(database).getNodeStore(); > Repository repo = new Jcr(new Oak(store)).createRepository(); > SimpleCredentials creds = new SimpleCredentials("admin", > "admin".toCharArray()); > Session session = (javax.jcr.Session) repository.login(creds); > Node parent = session.getRootNode().getNode(parentPath); > Node child = parent.addNode(childName, "sling:Folder"); > session.save(); > session.logout(); > store.dispose(); > } > > // Test case > public void test1() { > //Make POST request to createChildFolder using HttpURLConnection > //Get back JSON response by making GET request to Sling at > localhost:8080 (which is persisted by MongoDB at 127.0.0.1:27017) using > HttpURLConnection > //Compare resulting JSON from GET request with one that has child > node at parent > } > > Thanks, > H > > > From: jus...@justinedelson.com > > Date: Tue, 1 Dec 2015 19:43:10 + > > Subject: Re: Sling retrieving out-of-date data > > To: users@sling.apache.org > > > > Are you saying that you have a non-Oak application which connects directly > > to MongoDB and manipulates the documents? > > > > On Tue, Dec 1, 2015 at 2:22 PM H K <redflagba...@hotmail.com> wrote: > > > > > Hi, > > > > > > Sure. In pseudocode, here it is: > > > > > > 1. Have Sling standalone running and have application that provides REST > > > service createChildFolder (for example sake). Application connects to > > > repository by using DocumentMK, as in the link from the OP post. > > > > > > 2. Call createChildFolder("pathToParent", "childName"). > > > > > > 3. Make GET request to http://localhost:8080/pathToParent.infinity.json > > > > > > 4. Inspecting the JSON Object, expect there to be a child node in parent > > > node but none are there. Test failed. > > > > > > However, if I let the thread sleep for 2 seconds after step 2, child > > > exists in the JSON as wanted. > > > > > > Thanks, > > > H > > > > > > > From: apa...@oliverlietz.de > > > > To: users@sling.apache.org > > > > Subject: Re: Sling retrieving out-of-date data > > > > Date: Tue, 1 Dec 2015 19:49:20 +0100 > > > > > > > > On Tuesday 01 December 2015 11:37:42 H K wrote: > > > > > Hi, > > > > > > > > Hi, > > > > > > > > > I'm currently using the Sling 8 standalone and it's connected to a > > > MongoD
Re: Sling retrieving out-of-date data
On Tue, Dec 1, 2015 at 10:07 PM, H Kwrote: > My problem is that when I write data to the repository through my > application, Sling does not seem to get it fast enough, which may be because > it is caching or because it only periodically checks to see if the data it > has is up-to-date or not What you are observing here is the result of eventual consistency model of Oak which is a MVCC based store [1]. When using DocumentNodeStore (used for Mongo) each cluster node periodically polls the backend for change in root node and then updates its head revision to make those changes visible to current cluster node. So change done by one cluster not would not be immediately *visible* on other cluster nodes. > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) and > connecting over RMI. That was because there you were not using a cluster. You were connecting to the JR running within Sling using RMI. So it was a single node setup. If you use RMI now also and connect to same Sling node then it would work as expected. Chetan Mehrotra [1] http://jackrabbit.apache.org/oak/docs/architecture/nodestate.html
Re: Sling retrieving out-of-date data
On Tuesday 01 December 2015 11:37:42 H K wrote: > Hi, Hi, > I'm currently using the Sling 8 standalone and it's connected to a MongoDB > database. I've got an application which also connects to the same MongoDB > database and writes/reads from it. The connection is done using DocumentMK, > following the instructions at > https://jackrabbit.apache.org/oak/docs/construct.html > > My problem is that when I write data to the repository through my > application, Sling does not seem to get it fast enough, which may be > because it is caching or because it only periodically checks to see if the > data it has is up-to-date or not. For instance in my unit tests that test > some of my REST services, I'm creating a child node somewhere and then > doing a GET request to Sling to get back a JSON representing the updated > parent. However, the returned result does not have the child in it. I know > the changes are made because if I delay for two seconds before submitting > the GET request, all works fine. > > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) and > connecting over RMI. > > Can someone provide some insight and/or a solution? Thanks! can you provide a minimal test case which resembles what you are doing? Regards, O.
RE: Sling retrieving out-of-date data
Hi, Sure. In pseudocode, here it is: 1. Have Sling standalone running and have application that provides REST service createChildFolder (for example sake). Application connects to repository by using DocumentMK, as in the link from the OP post. 2. Call createChildFolder("pathToParent", "childName"). 3. Make GET request to http://localhost:8080/pathToParent.infinity.json 4. Inspecting the JSON Object, expect there to be a child node in parent node but none are there. Test failed. However, if I let the thread sleep for 2 seconds after step 2, child exists in the JSON as wanted. Thanks, H > From: apa...@oliverlietz.de > To: users@sling.apache.org > Subject: Re: Sling retrieving out-of-date data > Date: Tue, 1 Dec 2015 19:49:20 +0100 > > On Tuesday 01 December 2015 11:37:42 H K wrote: > > Hi, > > Hi, > > > I'm currently using the Sling 8 standalone and it's connected to a MongoDB > > database. I've got an application which also connects to the same MongoDB > > database and writes/reads from it. The connection is done using DocumentMK, > > following the instructions at > > https://jackrabbit.apache.org/oak/docs/construct.html > > > > My problem is that when I write data to the repository through my > > application, Sling does not seem to get it fast enough, which may be > > because it is caching or because it only periodically checks to see if the > > data it has is up-to-date or not. For instance in my unit tests that test > > some of my REST services, I'm creating a child node somewhere and then > > doing a GET request to Sling to get back a JSON representing the updated > > parent. However, the returned result does not have the child in it. I know > > the changes are made because if I delay for two seconds before submitting > > the GET request, all works fine. > > > > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) and > > connecting over RMI. > > > > Can someone provide some insight and/or a solution? Thanks! > > can you provide a minimal test case which resembles what you are doing? > > Regards, > O. >
Re: Sling retrieving out-of-date data
Are you saying that you have a non-Oak application which connects directly to MongoDB and manipulates the documents? On Tue, Dec 1, 2015 at 2:22 PM H K <redflagba...@hotmail.com> wrote: > Hi, > > Sure. In pseudocode, here it is: > > 1. Have Sling standalone running and have application that provides REST > service createChildFolder (for example sake). Application connects to > repository by using DocumentMK, as in the link from the OP post. > > 2. Call createChildFolder("pathToParent", "childName"). > > 3. Make GET request to http://localhost:8080/pathToParent.infinity.json > > 4. Inspecting the JSON Object, expect there to be a child node in parent > node but none are there. Test failed. > > However, if I let the thread sleep for 2 seconds after step 2, child > exists in the JSON as wanted. > > Thanks, > H > > > From: apa...@oliverlietz.de > > To: users@sling.apache.org > > Subject: Re: Sling retrieving out-of-date data > > Date: Tue, 1 Dec 2015 19:49:20 +0100 > > > > On Tuesday 01 December 2015 11:37:42 H K wrote: > > > Hi, > > > > Hi, > > > > > I'm currently using the Sling 8 standalone and it's connected to a > MongoDB > > > database. I've got an application which also connects to the same > MongoDB > > > database and writes/reads from it. The connection is done using > DocumentMK, > > > following the instructions at > > > https://jackrabbit.apache.org/oak/docs/construct.html > > > > > > My problem is that when I write data to the repository through my > > > application, Sling does not seem to get it fast enough, which may be > > > because it is caching or because it only periodically checks to see if > the > > > data it has is up-to-date or not. For instance in my unit tests that > test > > > some of my REST services, I'm creating a child node somewhere and then > > > doing a GET request to Sling to get back a JSON representing the > updated > > > parent. However, the returned result does not have the child in it. I > know > > > the changes are made because if I delay for two seconds before > submitting > > > the GET request, all works fine. > > > > > > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) > and > > > connecting over RMI. > > > > > > Can someone provide some insight and/or a solution? Thanks! > > > > can you provide a minimal test case which resembles what you are doing? > > > > Regards, > > O. > > >
RE: Sling retrieving out-of-date data
It is an Oak application and yes, it does manipulate the documents directly without making REST calls to Sling. Much of it taken from http://jackrabbit.apache.org/oak/docs/construct.html , here is what the code may look like: // REST service createChildFolder running on localhost:8040 public void createChildFolder(String parentPath, String childName) { DB database = new MongoClient("127.0.0.1", 27017).getDB("sling"); DocumentNodeStore store = new DocumentMK.Builder().setMongoDB(database).getNodeStore(); Repository repo = new Jcr(new Oak(store)).createRepository(); SimpleCredentials creds = new SimpleCredentials("admin", "admin".toCharArray()); Session session = (javax.jcr.Session) repository.login(creds); Node parent = session.getRootNode().getNode(parentPath); Node child = parent.addNode(childName, "sling:Folder"); session.save(); session.logout(); store.dispose(); } // Test case public void test1() { //Make POST request to createChildFolder using HttpURLConnection //Get back JSON response by making GET request to Sling at localhost:8080 (which is persisted by MongoDB at 127.0.0.1:27017) using HttpURLConnection //Compare resulting JSON from GET request with one that has child node at parent } Thanks, H > From: jus...@justinedelson.com > Date: Tue, 1 Dec 2015 19:43:10 + > Subject: Re: Sling retrieving out-of-date data > To: users@sling.apache.org > > Are you saying that you have a non-Oak application which connects directly > to MongoDB and manipulates the documents? > > On Tue, Dec 1, 2015 at 2:22 PM H K <redflagba...@hotmail.com> wrote: > > > Hi, > > > > Sure. In pseudocode, here it is: > > > > 1. Have Sling standalone running and have application that provides REST > > service createChildFolder (for example sake). Application connects to > > repository by using DocumentMK, as in the link from the OP post. > > > > 2. Call createChildFolder("pathToParent", "childName"). > > > > 3. Make GET request to http://localhost:8080/pathToParent.infinity.json > > > > 4. Inspecting the JSON Object, expect there to be a child node in parent > > node but none are there. Test failed. > > > > However, if I let the thread sleep for 2 seconds after step 2, child > > exists in the JSON as wanted. > > > > Thanks, > > H > > > > > From: apa...@oliverlietz.de > > > To: users@sling.apache.org > > > Subject: Re: Sling retrieving out-of-date data > > > Date: Tue, 1 Dec 2015 19:49:20 +0100 > > > > > > On Tuesday 01 December 2015 11:37:42 H K wrote: > > > > Hi, > > > > > > Hi, > > > > > > > I'm currently using the Sling 8 standalone and it's connected to a > > MongoDB > > > > database. I've got an application which also connects to the same > > MongoDB > > > > database and writes/reads from it. The connection is done using > > DocumentMK, > > > > following the instructions at > > > > https://jackrabbit.apache.org/oak/docs/construct.html > > > > > > > > My problem is that when I write data to the repository through my > > > > application, Sling does not seem to get it fast enough, which may be > > > > because it is caching or because it only periodically checks to see if > > the > > > > data it has is up-to-date or not. For instance in my unit tests that > > test > > > > some of my REST services, I'm creating a child node somewhere and then > > > > doing a GET request to Sling to get back a JSON representing the > > updated > > > > parent. However, the returned result does not have the child in it. I > > know > > > > the changes are made because if I delay for two seconds before > > submitting > > > > the GET request, all works fine. > > > > > > > > This was not a problem using Sling 7 with Jackrabbit (instead of Oak) > > and > > > > connecting over RMI. > > > > > > > > Can someone provide some insight and/or a solution? Thanks! > > > > > > can you provide a minimal test case which resembles what you are doing? > > > > > > Regards, > > > O. > > > > >