Hi Ian, Once you look at the results, let me know whether you want new tests run results. Also please advice how to proceed further.
On Wed, Sep 11, 2013 at 7:59 PM, Dishara Wijewardana < [email protected]> wrote: > Hi Ian, > I attached the results.zip file to the JIRA which has the latest results > as your expectation. I added new set of nodes in reverse order "F", "E" > .."A". > But here I came across in a issue (which I didn't came up last time, may > be due to my heap size), that hector fails to execute batches > around > 300-500.. So in my case batch size is 600. I googled for the issue > (me.prettyprint.hector.api.exceptions.HectorException: All host pools > marked down. Retry burden pushed out to client) and the solution was reduce > the batch size. Hence I ran the test for each node separately. Hence In new > results zip file each node has its own report file. I categorized them in > two 3 main folders so that you can easily locate the results. i.,e folders > "_create", "_update" , "_delete" .. > > > > > > On Wed, Sep 11, 2013 at 1:58 PM, Ian Boston <[email protected]> wrote: > >> Hi, >> Could you re-run the tests for create in the reverse order, largest >> collection first. >> >> The reason for this is in the detail of the tests adding resources to >> collection E (10M child nodes) is flat until half way through and then >> starts to rise which makes me thing there might be something else >> going on on the machine of the create mechanism. I would like to rule >> out that Cassandra is caching writes and the speed of update bears no >> correlation to collection size, but is a function of sustained load. I >> would also like to rule out any possibility that another OS level >> process started to hog resources in the latter part of your tests. >> >> The results do look comparatively flat all the way up 100M items in a >> collection. >> >> Once we have done this we can move on to providing access control, on >> items. >> >> Best Regards >> Ian >> >> >> >> On 11 September 2013 03:37, Dishara Wijewardana <[email protected]> >> wrote: >> > Hi Ian, >> > Sorry the delay, I found my self some battery problems with my laptop. I >> > believe this is what you asked for. Here I added new 100 collections i.e >> > 1a,3a,5a etc in each old collections and did CUD. Please advice what I >> > should do next. >> > >> > CREATE >> > [RESULT] Average Latency Under Node A(1K) = 11 (ms) >> > [RESULT] Average Latency Under Node B(10K) = 9 (ms) >> > [RESULT] Average Latency Under Node C(100K) = 13 (ms) >> > [RESULT] Average Latency Under Node D(1M) = 23 (ms) >> > [RESULT] Average Latency Under Node E(10M) = 110 (ms) >> > [RESULT] Average Latency Under Node F(100M) = 306 (ms) >> > [FIRST RUN] #TOTAL CALLS = 600 Total Average Latency = 79 (ms) >> > >> > UPDATE >> > [RESULT] Average Latency Under Node A(1K) = 9 (ms) >> > [RESULT] Average Latency Under Node B(10K) = 8 (ms) >> > [RESULT] Average Latency Under Node C(100K) = 9 (ms) >> > [RESULT] Average Latency Under Node D(1M) = 17 (ms) >> > [RESULT] Average Latency Under Node E(10M) = 20 (ms) >> > [RESULT] Average Latency Under Node F(100M) = 26 (ms) >> > [FIRST RUN] #TOTAL CALLS = 600 Total Average Latency = 15 (ms) >> > >> > DELETE >> > [RESULT] Average Latency Under Node A(1K) = 12 (ms) >> > [RESULT] Average Latency Under Node B(10K) = 12 (ms) >> > [RESULT] Average Latency Under Node C(100K) = 11 (ms) >> > [RESULT] Average Latency Under Node C(100K) = 14 (ms) >> > [RESULT] Average Latency Under Node C(100K) = 19 (ms) >> > [RESULT] Average Latency Under Node C(100K) = 16 (ms) >> > [FIRST RUN] #TOTAL CALLS = 600 Total Average Latency = 14 (ms) >> > >> > >> > >> > On Tue, Sep 10, 2013 at 5:25 PM, Ian Boston <[email protected]> wrote: >> > >> >> Are they pre-populated with different numbers of items, or are they >> >> empty when the test starts ? >> >> >> >> If they are empty and created by the test, can you run the test on the >> >> collections used in the read tests so we get an idea how long it takes >> >> to add 100 items to each of those collections. >> >> >> >> Thanks >> >> Ian >> >> >> >> On 10 September 2013 12:25, Dishara Wijewardana < >> [email protected]> >> >> wrote: >> >> > Hi Ian, >> >> > Those are newly added collections. >> >> > >> >> > >> >> > On Tue, Sep 10, 2013 at 12:00 PM, Ian Boston <[email protected]> wrote: >> >> > >> >> >> Hi Dishara, >> >> >> Great, thanks. >> >> >> What do the LA, MA, and SA stand for ? >> >> >> Best regards >> >> >> Ian >> >> >> >> >> >> On Tuesday, September 10, 2013, Dishara Wijewardana wrote: >> >> >> >> >> >> > Hi Ian >> >> >> > Also commited the 3 new test classes for CUD to google code repo. >> >> >> > >> >> >> > >> >> >> > On Tue, Sep 10, 2013 at 10:30 AM, Dishara Wijewardana < >> >> >> > [email protected] <javascript:;>> wrote: >> >> >> > >> >> >> > > Hi Ian, >> >> >> > > Finally was able to fix ;-). Had to debug along with the path >> and >> >> fix >> >> >> > some >> >> >> > > other issues also in Cassandra Provider impl. I have updated the >> >> JIRA >> >> >> > with >> >> >> > > the new reports of CUD. >> >> >> > > (I ran them one after the other respectively) >> >> >> > > >> >> >> > > In brief as follows. >> >> >> > > CREATE >> >> >> > > Average Latency Under Node LA = 165 (ms) >> >> >> > > Average Latency Under Node MA = 203 (ms) >> >> >> > > Average Latency Under Node SA = 256 (ms) >> >> >> > > #TOTAL CALLS = 300 Total Average Latency = 208 (ms) >> >> >> > > >> >> >> > > UPDATE >> >> >> > > Average Latency Under Node LA = 36 (ms) >> >> >> > > Average Latency Under Node MA = 29 (ms) >> >> >> > > Average Latency Under Node SA = 25 (ms) >> >> >> > > #TOTAL CALLS = 300 Total Average Latency = 30 (ms) >> >> >> > > >> >> >> > > DELETE >> >> >> > > Average Latency Under Node LA = 20 (ms) >> >> >> > > Average Latency Under Node MA = 19 (ms) >> >> >> > > Average Latency Under Node SA = 19 (ms) >> >> >> > > #TOTAL CALLS = 300 Total Average Latency = 19 (ms) >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > On Mon, Sep 9, 2013 at 5:47 PM, Ian Boston <[email protected]> >> wrote: >> >> >> > > >> >> >> > >> Hi >> >> >> > >> The CassandraImpl needs to return something suitable to >> represent >> >> >> > >> >> >> >> > >> /content/cassandra/p >> >> >> > >> >> >> >> > >> For the moment, if the column family p does exist, return a new >> >> type >> >> >> > >> of read only resource, eg CassandraColumnFamilyResource. >> >> >> > >> >> >> >> > >> Ian >> >> >> > >> >> >> >> > >> >> >> >> > >> On 9 September 2013 13:02, Dishara Wijewardana < >> >> >> [email protected] >> >> >> > > >> >> >> > >> wrote: >> >> >> > >> > On Mon, Sep 9, 2013 at 2:09 PM, Ian Boston <[email protected]> >> >> wrote: >> >> >> > >> > >> >> >> > >> >> Hi, >> >> >> > >> >> The call to processCreate a few lines earlier should have >> >> created >> >> >> the >> >> >> > >> >> resource so that the subsequent call to getResource >> returned the >> >> >> new >> >> >> > >> >> resource into newResource. >> >> >> > >> >> >> >> >> > >> >> If you look through the calls the create operation is >> called in >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> > >> >> >> >> >> >> org.apache.sling.servlets.post.impl.operations.AbstractCreateOperation.deepGetOrCreateNode(ResourceResolver, >> >> >> > >> >> String, Map<String, RequestProperty>, List<Modification>, >> >> >> > >> >> VersioningConfiguration) >> >> >> > >> >> >> >> >> > >> >> Line 533 >> >> >> > >> >> >> >> >> > >> >> resource = resolver.create(resource, >> name, >> >> >> > props); >> >> >> > >> >> >> >> >> > >> >> Stick a breakpoint at that line and see, if its called, and >> if >> >> the >> >> >> > >> >> resource resolver calls your code. >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> > Hi Ian, >> >> >> > >> > Yes it calls the create method. As I see the issue is the >> commit >> >> not >> >> >> > >> > getting called and hence resolver cannot get the created >> >> resource. >> >> >> > >> > So I thought of commit on the fly @create method to verify >> that. >> >> But >> >> >> > we >> >> >> > >> > have a problem here. >> >> >> > >> > >> >> >> > >> > When I try to create /content/cassandra/p/c node, as you >> exactly >> >> >> > >> mentioned >> >> >> > >> > before, it tries to create /content/cassandra/p node. In >> >> Cassandra >> >> >> > >> Impl, >> >> >> > >> > there cannot be a node like /content/cassandra/p and hence >> >> returns >> >> >> > >> NULL as >> >> >> > >> > earlier and still create fails. Because "p" is the column >> family >> >> >> (the >> >> >> > >> model >> >> >> > >> > which we agreed and implemented). So what is the best >> approach to >> >> >> > >> overcome >> >> >> > >> > this. >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> >> BTW: Its Ok that there is a newResource.adaptTo(Node.class) >> as >> >> the >> >> >> > >> >> result is checked for Null a moment later. You do not need >> to >> >> >> support >> >> >> > >> >> adaptTo(Node.class), which is a special case for Jcr. >> >> >> > >> >> >> >> >> > >> >> Best Regards >> >> >> > >> >> Ian >> >> >> > >> >> >> >> >> > >> >> On 9 September 2013 06:05, Dishara Wijewardana < >> >> >> > >> [email protected]> >> >> >> > >> >> wrote: >> >> >> > >> >> > Hi Ian, >> >> >> > >> >> > I debug the servlet from line to line and locate the NPE >> that >> >> >> > causes >> >> >> > >> this >> >> >> > >-- >> >> >> > Thanks >> >> >> > /Dishara >> >> >> > >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Thanks >> >> > /Dishara >> >> >> > >> > >> > >> > -- >> > Thanks >> > /Dishara >> > > > > -- > Thanks > /Dishara > -- Thanks /Dishara
