Hi Ian, I attached the results.zip file to the JIRA which has the latest results as your expectation. I added new set of nodes in reverse order "F", "E" .."A". But here I came across in a issue (which I didn't came up last time, may be due to my heap size), that hector fails to execute batches > around 300-500.. So in my case batch size is 600. I googled for the issue (me.prettyprint.hector.api.exceptions.HectorException: All host pools marked down. Retry burden pushed out to client) and the solution was reduce the batch size. Hence I ran the test for each node separately. Hence In new results zip file each node has its own report file. I categorized them in two 3 main folders so that you can easily locate the results. i.,e folders "_create", "_update" , "_delete" ..
On Wed, Sep 11, 2013 at 1:58 PM, Ian Boston <i...@tfd.co.uk> wrote: > Hi, > Could you re-run the tests for create in the reverse order, largest > collection first. > > The reason for this is in the detail of the tests adding resources to > collection E (10M child nodes) is flat until half way through and then > starts to rise which makes me thing there might be something else > going on on the machine of the create mechanism. I would like to rule > out that Cassandra is caching writes and the speed of update bears no > correlation to collection size, but is a function of sustained load. I > would also like to rule out any possibility that another OS level > process started to hog resources in the latter part of your tests. > > The results do look comparatively flat all the way up 100M items in a > collection. > > Once we have done this we can move on to providing access control, on > items. > > Best Regards > Ian > > > > On 11 September 2013 03:37, Dishara Wijewardana <ddwijeward...@gmail.com> > wrote: > > Hi Ian, > > Sorry the delay, I found my self some battery problems with my laptop. I > > believe this is what you asked for. Here I added new 100 collections i.e > > 1a,3a,5a etc in each old collections and did CUD. Please advice what I > > should do next. > > > > CREATE > > [RESULT] Average Latency Under Node A(1K) = 11 (ms) > > [RESULT] Average Latency Under Node B(10K) = 9 (ms) > > [RESULT] Average Latency Under Node C(100K) = 13 (ms) > > [RESULT] Average Latency Under Node D(1M) = 23 (ms) > > [RESULT] Average Latency Under Node E(10M) = 110 (ms) > > [RESULT] Average Latency Under Node F(100M) = 306 (ms) > > [FIRST RUN] #TOTAL CALLS = 600 Total Average Latency = 79 (ms) > > > > UPDATE > > [RESULT] Average Latency Under Node A(1K) = 9 (ms) > > [RESULT] Average Latency Under Node B(10K) = 8 (ms) > > [RESULT] Average Latency Under Node C(100K) = 9 (ms) > > [RESULT] Average Latency Under Node D(1M) = 17 (ms) > > [RESULT] Average Latency Under Node E(10M) = 20 (ms) > > [RESULT] Average Latency Under Node F(100M) = 26 (ms) > > [FIRST RUN] #TOTAL CALLS = 600 Total Average Latency = 15 (ms) > > > > DELETE > > [RESULT] Average Latency Under Node A(1K) = 12 (ms) > > [RESULT] Average Latency Under Node B(10K) = 12 (ms) > > [RESULT] Average Latency Under Node C(100K) = 11 (ms) > > [RESULT] Average Latency Under Node C(100K) = 14 (ms) > > [RESULT] Average Latency Under Node C(100K) = 19 (ms) > > [RESULT] Average Latency Under Node C(100K) = 16 (ms) > > [FIRST RUN] #TOTAL CALLS = 600 Total Average Latency = 14 (ms) > > > > > > > > On Tue, Sep 10, 2013 at 5:25 PM, Ian Boston <i...@tfd.co.uk> wrote: > > > >> Are they pre-populated with different numbers of items, or are they > >> empty when the test starts ? > >> > >> If they are empty and created by the test, can you run the test on the > >> collections used in the read tests so we get an idea how long it takes > >> to add 100 items to each of those collections. > >> > >> Thanks > >> Ian > >> > >> On 10 September 2013 12:25, Dishara Wijewardana < > ddwijeward...@gmail.com> > >> wrote: > >> > Hi Ian, > >> > Those are newly added collections. > >> > > >> > > >> > On Tue, Sep 10, 2013 at 12:00 PM, Ian Boston <i...@tfd.co.uk> wrote: > >> > > >> >> Hi Dishara, > >> >> Great, thanks. > >> >> What do the LA, MA, and SA stand for ? > >> >> Best regards > >> >> Ian > >> >> > >> >> On Tuesday, September 10, 2013, Dishara Wijewardana wrote: > >> >> > >> >> > Hi Ian > >> >> > Also commited the 3 new test classes for CUD to google code repo. > >> >> > > >> >> > > >> >> > On Tue, Sep 10, 2013 at 10:30 AM, Dishara Wijewardana < > >> >> > ddwijeward...@gmail.com <javascript:;>> wrote: > >> >> > > >> >> > > Hi Ian, > >> >> > > Finally was able to fix ;-). Had to debug along with the path and > >> fix > >> >> > some > >> >> > > other issues also in Cassandra Provider impl. I have updated the > >> JIRA > >> >> > with > >> >> > > the new reports of CUD. > >> >> > > (I ran them one after the other respectively) > >> >> > > > >> >> > > In brief as follows. > >> >> > > CREATE > >> >> > > Average Latency Under Node LA = 165 (ms) > >> >> > > Average Latency Under Node MA = 203 (ms) > >> >> > > Average Latency Under Node SA = 256 (ms) > >> >> > > #TOTAL CALLS = 300 Total Average Latency = 208 (ms) > >> >> > > > >> >> > > UPDATE > >> >> > > Average Latency Under Node LA = 36 (ms) > >> >> > > Average Latency Under Node MA = 29 (ms) > >> >> > > Average Latency Under Node SA = 25 (ms) > >> >> > > #TOTAL CALLS = 300 Total Average Latency = 30 (ms) > >> >> > > > >> >> > > DELETE > >> >> > > Average Latency Under Node LA = 20 (ms) > >> >> > > Average Latency Under Node MA = 19 (ms) > >> >> > > Average Latency Under Node SA = 19 (ms) > >> >> > > #TOTAL CALLS = 300 Total Average Latency = 19 (ms) > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > On Mon, Sep 9, 2013 at 5:47 PM, Ian Boston <i...@tfd.co.uk> > wrote: > >> >> > > > >> >> > >> Hi > >> >> > >> The CassandraImpl needs to return something suitable to > represent > >> >> > >> > >> >> > >> /content/cassandra/p > >> >> > >> > >> >> > >> For the moment, if the column family p does exist, return a new > >> type > >> >> > >> of read only resource, eg CassandraColumnFamilyResource. > >> >> > >> > >> >> > >> Ian > >> >> > >> > >> >> > >> > >> >> > >> On 9 September 2013 13:02, Dishara Wijewardana < > >> >> ddwijeward...@gmail.com > >> >> > > > >> >> > >> wrote: > >> >> > >> > On Mon, Sep 9, 2013 at 2:09 PM, Ian Boston <i...@tfd.co.uk> > >> wrote: > >> >> > >> > > >> >> > >> >> Hi, > >> >> > >> >> The call to processCreate a few lines earlier should have > >> created > >> >> the > >> >> > >> >> resource so that the subsequent call to getResource returned > the > >> >> new > >> >> > >> >> resource into newResource. > >> >> > >> >> > >> >> > >> >> If you look through the calls the create operation is called > in > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> > >> >> > > >> >> > >> > org.apache.sling.servlets.post.impl.operations.AbstractCreateOperation.deepGetOrCreateNode(ResourceResolver, > >> >> > >> >> String, Map<String, RequestProperty>, List<Modification>, > >> >> > >> >> VersioningConfiguration) > >> >> > >> >> > >> >> > >> >> Line 533 > >> >> > >> >> > >> >> > >> >> resource = resolver.create(resource, > name, > >> >> > props); > >> >> > >> >> > >> >> > >> >> Stick a breakpoint at that line and see, if its called, and > if > >> the > >> >> > >> >> resource resolver calls your code. > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> > Hi Ian, > >> >> > >> > Yes it calls the create method. As I see the issue is the > commit > >> not > >> >> > >> > getting called and hence resolver cannot get the created > >> resource. > >> >> > >> > So I thought of commit on the fly @create method to verify > that. > >> But > >> >> > we > >> >> > >> > have a problem here. > >> >> > >> > > >> >> > >> > When I try to create /content/cassandra/p/c node, as you > exactly > >> >> > >> mentioned > >> >> > >> > before, it tries to create /content/cassandra/p node. In > >> Cassandra > >> >> > >> Impl, > >> >> > >> > there cannot be a node like /content/cassandra/p and hence > >> returns > >> >> > >> NULL as > >> >> > >> > earlier and still create fails. Because "p" is the column > family > >> >> (the > >> >> > >> model > >> >> > >> > which we agreed and implemented). So what is the best > approach to > >> >> > >> overcome > >> >> > >> > this. > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > >> >> BTW: Its Ok that there is a newResource.adaptTo(Node.class) > as > >> the > >> >> > >> >> result is checked for Null a moment later. You do not need to > >> >> support > >> >> > >> >> adaptTo(Node.class), which is a special case for Jcr. > >> >> > >> >> > >> >> > >> >> Best Regards > >> >> > >> >> Ian > >> >> > >> >> > >> >> > >> >> On 9 September 2013 06:05, Dishara Wijewardana < > >> >> > >> ddwijeward...@gmail.com> > >> >> > >> >> wrote: > >> >> > >> >> > Hi Ian, > >> >> > >> >> > I debug the servlet from line to line and locate the NPE > that > >> >> > causes > >> >> > >> this > >> >> > >-- > >> >> > Thanks > >> >> > /Dishara > >> >> > > >> >> > >> > > >> > > >> > > >> > -- > >> > Thanks > >> > /Dishara > >> > > > > > > > > -- > > Thanks > > /Dishara > -- Thanks /Dishara
