Hi Ian Also commited the 3 new test classes for CUD to google code repo.
On Tue, Sep 10, 2013 at 10:30 AM, Dishara Wijewardana < ddwijeward...@gmail.com> wrote: > Hi Ian, > Finally was able to fix ;-). Had to debug along with the path and fix some > other issues also in Cassandra Provider impl. I have updated the JIRA with > the new reports of CUD. > (I ran them one after the other respectively) > > In brief as follows. > CREATE > Average Latency Under Node LA = 165 (ms) > Average Latency Under Node MA = 203 (ms) > Average Latency Under Node SA = 256 (ms) > #TOTAL CALLS = 300 Total Average Latency = 208 (ms) > > UPDATE > Average Latency Under Node LA = 36 (ms) > Average Latency Under Node MA = 29 (ms) > Average Latency Under Node SA = 25 (ms) > #TOTAL CALLS = 300 Total Average Latency = 30 (ms) > > DELETE > Average Latency Under Node LA = 20 (ms) > Average Latency Under Node MA = 19 (ms) > Average Latency Under Node SA = 19 (ms) > #TOTAL CALLS = 300 Total Average Latency = 19 (ms) > > > > > > > > On Mon, Sep 9, 2013 at 5:47 PM, Ian Boston <i...@tfd.co.uk> wrote: > >> Hi >> The CassandraImpl needs to return something suitable to represent >> >> /content/cassandra/p >> >> For the moment, if the column family p does exist, return a new type >> of read only resource, eg CassandraColumnFamilyResource. >> >> Ian >> >> >> On 9 September 2013 13:02, Dishara Wijewardana <ddwijeward...@gmail.com> >> wrote: >> > On Mon, Sep 9, 2013 at 2:09 PM, Ian Boston <i...@tfd.co.uk> wrote: >> > >> >> Hi, >> >> The call to processCreate a few lines earlier should have created the >> >> resource so that the subsequent call to getResource returned the new >> >> resource into newResource. >> >> >> >> If you look through the calls the create operation is called in >> >> >> >> >> >> >> org.apache.sling.servlets.post.impl.operations.AbstractCreateOperation.deepGetOrCreateNode(ResourceResolver, >> >> String, Map<String, RequestProperty>, List<Modification>, >> >> VersioningConfiguration) >> >> >> >> Line 533 >> >> >> >> resource = resolver.create(resource, name, props); >> >> >> >> Stick a breakpoint at that line and see, if its called, and if the >> >> resource resolver calls your code. >> >> >> >> >> >> >> > Hi Ian, >> > Yes it calls the create method. As I see the issue is the commit not >> > getting called and hence resolver cannot get the created resource. >> > So I thought of commit on the fly @create method to verify that. But we >> > have a problem here. >> > >> > When I try to create /content/cassandra/p/c node, as you exactly >> mentioned >> > before, it tries to create /content/cassandra/p node. In Cassandra >> Impl, >> > there cannot be a node like /content/cassandra/p and hence returns >> NULL as >> > earlier and still create fails. Because "p" is the column family (the >> model >> > which we agreed and implemented). So what is the best approach to >> overcome >> > this. >> > >> > >> > >> >> BTW: Its Ok that there is a newResource.adaptTo(Node.class) as the >> >> result is checked for Null a moment later. You do not need to support >> >> adaptTo(Node.class), which is a special case for Jcr. >> >> >> >> Best Regards >> >> Ian >> >> >> >> On 9 September 2013 06:05, Dishara Wijewardana < >> ddwijeward...@gmail.com> >> >> wrote: >> >> > Hi Ian, >> >> > I debug the servlet from line to line and locate the NPE that causes >> this >> >> > failure. The request path >> >> > >> >> > SlingPostServlet > AbstractCreateOperation > ModifyOperation@line105 >> >> > where newResource = null. The resource resolver at this point >> >> > is org.apache.sling.resourceresolver.impl.ResourceResolverImpl. >> >> > >> >> > 104 final Resource newResource = >> >> > request.getResourceResolver().getResource(response.getPath()); >> >> > 105 final Node newNode = *newResource*.adaptTo(Node.class); >> >> > >> >> > The resolver get a resource if the resource is already existing >> >> > one(obviously). So either this should be a bug or my request goes >> through >> >> > the wrong path. Please advice. >> >> > >> >> > >> >> > >> >> > >> >> > On Mon, Sep 9, 2013 at 8:45 AM, Dishara Wijewardana < >> >> ddwijeward...@gmail.com >> >> >> wrote: >> >> > >> >> >> I also checked in Cassandra Provider impl and with system logs and >> >> >> verified the Error not comes due to exception throws from the >> Provider >> >> >> implementation itself. >> >> >> So probably it will be a missing a property. I still could not >> figure >> >> out >> >> >> which. If I can find which servlet get called I can debug and >> >> see(already >> >> >> post a mail to dev list). >> >> >> >> >> >> >> >> >> On Sun, Sep 8, 2013 at 5:10 AM, Dishara Wijewardana < >> >> >> ddwijeward...@gmail.com> wrote: >> >> >> >> >> >>> Hi Ian, >> >> >>> I found it difficult to use createNode method to work. It returns >> 200 >> >> >>> response. I was trying to get this working but could not find a >> way. >> >> >>> Probably I might be missing some properties. What I set was only >> >> content >> >> >>> type to be json. What are the appropriate properties that you >> >> mentioned. >> >> >>> How can someone get to know what are the properties required >> minimum to >> >> >>> create a node. I debug and verified my create method gets hit in >> >> provider >> >> >>> side. >> >> >>> >> >> >>> *P.S If I give a already existing path it works and gives 302 >> >> response. * >> >> >>> i.e /content/cassandra/p1/c1 this works since it is already there. >> >> >>> But /content/cassandra/pp/cc fails. I also tried >> >> >>> adding /content/cassandra/pp first and then >> /content/cassandra/pp/cc. >> >> It >> >> >>> fails at /content/cassandra/pp. >> >> >>> >> >> >>> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: >> 36.21 >> >> sec >> >> >>> <<< FAILURE! >> >> >>> >> >> >> testResourceCreate(org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceCUDTest) >> >> >>> Time elapsed: 36.187 sec <<< ERROR! >> >> >>> >> org.apache.sling.commons.testing.integration.HttpStatusCodeException: >> >> >>> Expected status code 302 for POST, got 200, URL= >> >> >>> http://localhost:8080/content/cassandra/pp/cc, Content=[<html> >> >> >>> <head> >> >> >>> <title>Error while processing /content/cassandra/pp/cc</title> >> >> >>> </head> >> >> >>> <body> >> >> >>> <h1>Error while processing /content/cassandra/pp/cc</h1> >> >> >>> <table> >> >> >>> <tbody> >> >> >>> <tr> >> >> >>> <td>Status</td> >> >> >>> <td><div id="Status">500</div></td> >> >> >>> </tr> >> >> >>> <tr> >> >> >>> <td>Message</td> >> >> >>> <td><div >> >> >>> id="Message">java.lang.NullPointerException</div></td> >> >> >>> </tr> >> >> >>> <tr> >> >> >>> <td>Location</td> >> >> >>> <td><a href="/cassandra/pp/cc" >> >> >>> id="Location">/cassandra/pp/cc</a></td> >> >> >>> </tr> >> >> >>> <tr> >> >> >>> <td>Parent Location</td> >> >> >>> <td><a href="/cassandra/pp" >> >> >>> id="ParentLocation">/cassandra/pp</a></td> >> >> >>> </tr> >> >> >>> <tr> >> >> >>> <td>Path</td> >> >> >>> <td><div >> id="Path">/content/cassandra/pp/cc</div></td> >> >> >>> </tr> >> >> >>> <tr> >> >> >>> <td>Referer</td> >> >> >>> <td><a href="" id="Referer"></a></td> >> >> >>> </tr> >> >> >>> <tr> >> >> >>> <td>ChangeLog</td> >> >> >>> <td><div >> >> id="ChangeLog"><pre></pre></div></td> >> >> >>> </tr> >> >> >>> </tbody> >> >> >>> </table> >> >> >>> <p><a href="">Go Back</a></p> >> >> >>> <p><a href="/cassandra/pp/cc">Modified Resource</a></p> >> >> >>> <p><a href="/cassandra/pp">Parent of Modified Resource</a></p> >> >> >>> </body> >> >> >>> </html>] >> >> >>> at >> >> >>> >> >> >> org.apache.sling.commons.testing.integration.SlingIntegrationTestClient.createNode(SlingIntegrationTestClient.java:188) >> >> >>> at >> >> >>> >> >> >> org.apache.sling.commons.testing.integration.SlingIntegrationTestClient.createNode(SlingIntegrationTestClient.java:119) >> >> >>> at >> >> >>> >> >> >> org.apache.sling.commons.testing.integration.SlingIntegrationTestClient.createNode(SlingIntegrationTestClient.java:109) >> >> >>> at >> >> >>> >> >> >> org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceCUDTest.testResourceCreate(CassandraResourceCUDTest.java:29) >> >> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> >>> at >> >> >>> >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> >> >>> at >> >> >>> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> >> >>> at java.lang.reflect.Method.invoke(Method.java:597) >> >> >>> at junit.framework.TestCase.runTest(TestCase.java:168) >> >> >>> at junit.framework.TestCase.runBare(TestCase.java:134) >> >> >>> at junit.framework.TestResult$1.protect(TestResult.java:110) >> >> >>> at junit.framework.TestResult.runProtected(TestResult.java:128) >> >> >>> at junit.framework.TestResult.run(TestResult.java:113) >> >> >>> at junit.framework.TestCase.run(TestCase.java:124) >> >> >>> at junit.framework.TestSuite.runTest(TestSuite.java:232) >> >> >>> at junit.framework.TestSuite.run(TestSuite.java:227) >> >> >>> at >> >> >>> >> >> >> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) >> >> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> >>> at >> >> >>> >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> >> >>> at >> >> >>> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> >> >>> at java.lang.reflect.Method.invoke(Method.java:597) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) >> >> >>> at >> >> >>> >> >> >> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) >> >> >>> >> >> >>> >> >> >>> Results : >> >> >>> >> >> >>> Tests in error: >> >> >>> >> >> >>> >> >> >> testResourceCreate(org.apache.sling.launchpad.webapp.integrationtest.resourceprovider.CassandraResourceCUDTest): >> >> >>> Expected status code 302 for POST, got 200, URL= >> >> >>> http://localhost:8080/content/cassandra/pp/cc, Content=[<html>(..) >> >> >>> >> >> >>> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0 >> >> >>> >> >> >>> [INFO] >> >> >>> >> >> >> ------------------------------------------------------------------------ >> >> >>> [INFO] BUILD FAILURE >> >> >>> [INFO] >> >> >>> >> >> >> ------------------------------------------------------------------------ >> >> >>> [INFO] Total time: 40.126s >> >> >>> [INFO] Finished at: Sun Sep 08 05:05:47 IST 2013 >> >> >>> [INFO] Final Memory: 13M/490M >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> On Fri, Sep 6, 2013 at 1:23 PM, Ian Boston <i...@tfd.co.uk> wrote: >> >> >>> >> >> >>>> Hi, >> >> >>>> Thanks, now I see how you're doing it. Its good. >> >> >>>> >> >> >>>> So to do a Write (CUD[1]) test, create another class that extends >> >> >>>> HttpBaseTest and write a test method that invokes >> >> >>>> >> >> >>>> testClient.createNode(...) with an appropriate set of properties >> and a >> >> >>>> collection managed by the CassandraProvider. You will probably >> need to >> >> >>>> create that collection first, and its parent, or better still, >> reuse >> >> >>>> the existing pre-populated collections since we are looking for >> the >> >> >>>> scalability of CUD relative to collection size. >> >> >>>> >> >> >>>> I think if you use createNode on an existing node it will update >> it, >> >> >>>> and there is a delete method as well. >> >> >>>> >> >> >>>> testClient is a >> >> >>>> >> >> org.apache.sling.commons.testing.integration.SlingIntegrationTestClient >> >> >>>> and is protected in HttpBaseTest so you should have access. >> >> >>>> >> >> >>>> As I said, perform CUD on 100 new nodes in each collection and >> print >> >> >>>> the results out. >> >> >>>> >> >> >>>> Obviously add all new code to you repository so its all in one >> place. >> >> >>>> >> >> >>>> Best Regards >> >> >>>> Ian >> >> >>>> >> >> >>>> 1 Create Update Delete (you've done read, we need stats for each >> of >> >> >>>> these operations) >> >> >>>> >> >> >>>> >> >> >>>> On 6 September 2013 04:31, Dishara Wijewardana < >> >> ddwijeward...@gmail.com> >> >> >>>> wrote: >> >> >>>> > Hi Ian, >> >> >>>> > Done. >> >> >>>> > >> >> >>>> > >> >> >>>> > On Thu, Sep 5, 2013 at 10:53 AM, Ian Boston <i...@tfd.co.uk> >> wrote: >> >> >>>> > >> >> >>>> >> Hi Dishara, >> >> >>>> >> >> >> >>>> >> On Thursday, September 5, 2013, Dishara Wijewardana wrote: >> >> >>>> >> >> >> >>>> >> > Hi Ian, >> >> >>>> >> > Did you mean the sling integration test (which runs the >> >> performance >> >> >>>> tests >> >> >>>> >> > and provides reports) to move inside google code repo ? >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> Yes please. >> >> >>>> >> Thanks >> >> >>>> >> >> >> >>>> >> Best regards >> >> >>>> >> Ian >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> > >> >> >>>> >> > >> >> >>>> >> > On Wed, Sep 4, 2013 at 5:41 PM, Ian Boston <i...@tfd.co.uk> >> >> wrote: >> >> >>>> >> > >> >> >>>> >> > > On 4 September 2013 13:42, Dishara Wijewardana < >> >> >>>> >> ddwijeward...@gmail.com> >> >> >>>> >> > > wrote: >> >> >>>> >> > > > HI Ian, >> >> >>>> >> > > > >> >> >>>> >> > > > >> >> >>>> >> > > > >> >> >>>> >> > > > On Wed, Sep 4, 2013 at 4:00 PM, Ian Boston < >> i...@tfd.co.uk> >> >> >>>> wrote: >> >> >>>> >> > > > >> >> >>>> >> > > >> On 4 September 2013 12:20, Dishara Wijewardana < >> >> >>>> >> > ddwijeward...@gmail.com >> >> >>>> >> > > > >> >> >>>> >> > > >> wrote: >> >> >>>> >> > > >> <snip> >> >> >>>> >> > > >> >> >> >>>> >> > > >> >> How long does it take to add, update and delete, 100 >> new >> >> >>>> items to >> >> >>>> >> > the >> >> >>>> >> > > >> >> existing collections that you used for the read >> tests. If >> >> >>>> its >> >> >>>> >> > quick, >> >> >>>> >> > > you >> >> >>>> >> > > >> >> may need to make that 1000 new items. >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> > Hi Ian, >> >> >>>> >> > > >> > >> >> >>>> >> > > >> > In the modify provider interface there is no "update" >> >> >>>> method. Am I >> >> >>>> >> > > >> missing >> >> >>>> >> > > >> > something? >> >> >>>> >> > > >> >> >> >>>> >> > > >> You need to support adapTo(ModifyableValueMap.class) >> IIRC. >> >> >>>> >> > > >> >> >> >>>> >> > > > I saw the interface ModifyableValueMap, but could not >> find a >> >> >>>> clear >> >> >>>> >> > > example >> >> >>>> >> > > > usage of it as in PlanetResource case. But will see >> further. >> >> >>>> >> > > > >> >> >>>> >> > > >> >> >>>> >> > > IIRC the MongoDB ResourceProvider allows modification. >> grep the >> >> >>>> Sling >> >> >>>> >> > > source code for it. >> >> >>>> >> > > >> >> >>>> >> > > >> >> >>>> >> > > >> >> >> >>>> >> > > >> > >> >> >>>> >> > > >> > Can you please elaborate more on exact scope and >> goals on >> >> >>>> what is >> >> >>>> >> > > >> expected >> >> >>>> >> > > >> > from the new performance test class?Is it to capture >> the >> >> >>>> >> add/delete >> >> >>>> >> > > >> latency >> >> >>>> >> > > >> > on nodes added in each collection. >> >> >>>> >> > > >> >> >> >>>> >> > > >> yes. >> >> >>>> >> > > >> >> >> >>>> >> > > >> > >> >> >>>> >> > > >> > If so, is it possible to test add/delete extending >> >> HTTPBase >> >> >>>> test >> >> >>>> >> as >> >> >>>> >> > > done >> >> >>>> >> > > >> > for reading ? Or any other class that provides >> abstract >> >> test >> >> >>>> >> class/ >> >> >>>> >> > > APIs >> >> >>>> >> > > >> to >> >> >>>> >> > > >> > add/delete ? >> >> >>>> >> > > >> > >> >> >>>> >> > > >> >> >> >>>> >> > > >> Where are the classes that you used to perform the read >> >> tests, >> >> >>>> I >> >> >>>> >> cant >> >> >>>> >> > > >> see them in the repo ? >> >> >>>> >> > > >> >> >> >>>> >> > > >> The class that I used was >> >> >>>> >> > > > >> org.apache.sling.commons.testing.integration.HttpTestBase . I >> >> >>>> have >> >> >>>> >> > > attached >> >> >>>> >> > > > the patch of my class which extends this class. There I >> am >> >> usinf >> >> >>>> >> > > > getContent() method of the base class to do HTTP get >> content >> >> >>>> >> > operations. >> >> >>>> >> > > >> >> >>>> >> > > >> >> >>>> >> > > Could we get your test classes to run the load test into >> your >> >> >>>> >> > > repository please, even if its something that has to be run >> >> >>>> manually. >> >> >>>> >> > > >> >> >>>> >> > > Thanks >> >> >>>> >> > > >> >> >>>> >> > > >> >> >>>> >> > > Best Regards >> >> >>>> >> > > Ian >> >> >>>> >> > > >> >> >>>> >> > > >> >> >>>> >> > > > >> >> >>>> >> > > >> Ian >> >> >>>> >> > > >> >> >> >>>> >> > > >> >> >> >>>> >> > > >> >> >> >>>> >> > > >> > >> >> >>>> >> > > >> > >> >> >>>> >> > > >> >> Best regards >> >> >>>> >> > > >> >> Ian >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> >> >> >> >>>> >> > > >> >> > >> >> >>>> >> > > >> >> > >> >> >>>> >> > > >> >> > > Best Regards >> >> >>>> >> > > >> >> > > Ian >> >> >>>> >> > > >> >> > > >> >> >>>> >> > > >> >> > > On 3 September 2013 01:23, Dishara Wijewardana < >> >> >>>> >> > > >> >> ddwijeward...@gmail.com<javascript:;> >> >> >>>> >> > > >> >> > > >> >> >>>> >> > > >> >> > > wrote: >> >> >>>> >> > > >> >> > > > Hi Ian, >> >> >>>> >> > > >> >> > > > Thank you for the references. I just commited >> the >> >> >>>> changes. >> >> >>>> >> > > There I >> >> >>>> >> > > >> >> have >> >> >>>> >> > > >> >> > > > completed the whole implementation of >> >> >>>> >> > ModifyingResourceProvider >> >> >>>> >> > > >> >> > interface >> >> >>>> >> > > >> >> > > > for Cassandra Resource Provider (create, >> delete, >> >> >>>> commit , >> >> >>>> >> > > >> rollback >> >> >>>> >> > > >> >> > etc) >> >> >>>> >> > > >> >> > > > and wrote 3 test classes ( >> >> >>>> >> > > >> CassandraModifyResourceProvide**rAddTest, >> >> >>>> >> > > >> >> > > Cassan >> >> >>>> >> > > >> >> > > > draModif-- >> >> >>>> >> > Thanks >> >> >>>> >> > /Dishara >> >> >>>> >> > >> >> >>>> >> >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > -- >> >> >>>> > Thanks >> >> >>>> > /Dishara >> >> >>>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Thanks >> >> >>> /Dishara >> >> >>> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Thanks >> >> >> /Dishara >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Thanks >> >> > /Dishara >> >> >> > >> > >> > >> > -- >> > Thanks >> > /Dishara >> > > > > -- > Thanks > /Dishara > -- Thanks /Dishara
