If the record is available locally, there is no need to reload data from other nodes.
ie. LOCAL UPDATE -> inform client err/success (0.0009s) -> REPLICATE changes to other nodes (1.2923s) Am I missing something? Regards, - Ben On Friday, September 26, 2014 7:55:40 PM UTC+2, Lvc@ wrote: > > UPDATE can't be faster, because it load the current version of record to > restore in case the quorum is not reached. > > Lvc@ > > ᐧ > > On 25 September 2014 11:01, Luca Garulli <[email protected] <javascript:>> > wrote: > >> Hey Ben, >> This is also because your issue and reports. I'm looking what's wrong >> with UPDATE right now... >> >> Lvc@ >> >> ᐧ >> >> On 24 September 2014 23:41, <[email protected] <javascript:>> wrote: >> >>> Many thanks to Luca for seriously looking into this matter. >>> >>> I can report that inserts and deletes are blazingly fast compared to >>> what was previously. >>> >>> Only connects and updates seem to still lag behind. >>> >>> Following is an overview issuing commands over the binary interface: >>> >>> *dserver.sh* with *one* node up: >>> >>> connect: 0.002266 s >>> insert: 0.000607 s >>> update:0.000972 s >>> delete: 0.000280 s >>> >>> *dserver.sh* with *two* nodes up: >>> >>> connect:* 0.118694 s* >>> insert: *0.000548 s* <--- MASSIVE IMPROVEMENT >>> update: *1.292313 s* >>> delete: *0.000533 s* <--- MASSIVE IMPROVEMENT >>> >>> >>> With these results I look very forward to the final release of OrientDB >>> 2.0. Hopefully the connects and updates can be improved by then too. >>> >>> Regards, >>> - Ben >>> >>> >>> >>> On Wednesday, September 24, 2014 8:08:38 PM UTC+2, Lvc@ wrote: >>>> >>>> Hi Stuart, >>>> You're right. I've applied your fix, pushed, deployed on sonatype as >>>> 2.0-M2-SNAPSHOT. >>>> >>>> Lvc@ >>>> >>>> ᐧ >>>> >>>> On 24 September 2014 18:17, Stuart McCulloch <[email protected]> wrote: >>>> >>>>> On 24 Sep 2014, at 16:31, Luca Garulli <[email protected]> wrote: >>>>> >>>>> Hi guys, >>>>> I've some updates. Using an queue in the middle improved performance. >>>>> Furthermore I've extended the asynchronous also to transactions. >>>>> >>>>> Please could you re-test last version 2.0-M2-SNAPSHOT? >>>>> >>>>> >>>>> FYI, I get the following error using the latest snapshot in embedded >>>>> mode with Java 7: >>>>> >>>>> java.lang.NoSuchMethodError: java.util.concurrent. >>>>> ConcurrentHashMap.keySet()Ljava/util/concurrent/ >>>>> ConcurrentHashMap$KeySetView; >>>>> >>>>> I assume this is because the snapshot was built using Java 8, which >>>>> changed the return type of keySet for ConcurrentHashMap: >>>>> >>>>> http://docs.oracle.com/javase/8/docs/api/java/util/ >>>>> concurrent/ConcurrentHashMap.html#keySet-- >>>>> >>>>> whereas in Java 7 it’s still the plain Set: >>>>> >>>>> http://docs.oracle.com/javase/7/docs/api/java/util/ >>>>> concurrent/ConcurrentHashMap.html#keySet() >>>>> >>>>> Possible workarounds (assuming the plan is still to support Java 7) is >>>>> either build with a version of Java before 8, or use Map or ConcurrentMap >>>>> for the field rather than the concrete type: >>>>> >>>>> diff --git a/core/src/main/java/com/orientechnologies/orient/core/ >>>>> config/OContextConfiguration.java b/core/src/main/java/com/ >>>>> orientechnologies/orient/core/config/OContextConfiguration.java >>>>> index d0dee2b..52c28b4 100644 >>>>> --- a/core/src/main/java/com/orientechnologies/orient/core/ >>>>> config/OContextConfiguration.java >>>>> +++ b/core/src/main/java/com/orientechnologies/orient/core/ >>>>> config/OContextConfiguration.java >>>>> @@ -27,7 +27,7 @@ import java.util.concurrent.ConcurrentHashMap; >>>>> * >>>>> */ >>>>> public class OContextConfiguration implements Serializable { >>>>> - private final ConcurrentHashMap<String, Object> config = new >>>>> ConcurrentHashMap<String, Object>(); >>>>> + private final Map<String, Object> config = new >>>>> ConcurrentHashMap<String, Object>(); >>>>> >>>>> /** >>>>> * Empty constructor to create just a proxy for the >>>>> OGlobalConfiguration. No values are setted. >>>>> >>>>> ( this is the place where the exception originates, there may be a >>>>> couple of other fields using ConcurrentHashMap where they could just as >>>>> well use Map or ConcurrentMap... ) >>>>> >>>>> — >>>>> Cheers, Stuart >>>>> >>>>> Lvc@ >>>>> >>>>> On 24 September 2014 11:31, Luca Garulli <[email protected]> wrote: >>>>> >>>>>> Hi Ben, >>>>>> I found a bottleneck in Hazelcast: https://github.com/ >>>>>> hazelcast/hazelcast/issues/3661. >>>>>> >>>>>> I could put a queue between ODistributedStorage and HZ. Let me try it >>>>>> now. Soon a new push. >>>>>> >>>>>> Lvc@ >>>>>> >>>>>> On 24 September 2014 10:59, <[email protected]> wrote: >>>>>> >>>>>>> Hi Luca, >>>>>>> >>>>>>> I tested this and now get 0.24sec average inserts, which is the >>>>>>> fastest distributed performance I've experienced yet. >>>>>>> >>>>>>> There still seems to be communication with the remote node before >>>>>>> releasing the client though. >>>>>>> >>>>>>> Regards, >>>>>>> - Ben >>>>>>> >>>>>>> On Wednesday, September 24, 2014 1:24:23 AM UTC+2, Lvc@ wrote: >>>>>>>> >>>>>>>> Hi Ben, >>>>>>>> I've pushed a new version that reads the configuration from file >>>>>>>> *default-distributed-db-config.json*, by adding: >>>>>>>> >>>>>>>> * "executionMode": "asynchronous",* >>>>>>>> >>>>>>>> Lvc@ >>>>>>>> >>>>>>>> On 23 September 2014 21:53, <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Luca, >>>>>>>>> >>>>>>>>> Yes, I'm aware of that... >>>>>>>>> >>>>>>>>> My reference to 1.7.9 was only concerning Christian's suggestion. >>>>>>>>> >>>>>>>>> When I tested 2.0-M2, i still have no improvement. Does it need to >>>>>>>>> be activated for use in the console or binary interface? >>>>>>>>> >>>>>>>>> I do not have Java apps for creating documents. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> - Ben >>>>>>>>> >>>>>>>>> On Tuesday, September 23, 2014 9:14:57 PM UTC+2, Lvc@ wrote: >>>>>>>>>> >>>>>>>>>> Hi Ben, >>>>>>>>>> This is supported only in 2.0-M2. >>>>>>>>>> >>>>>>>>>> Lvc@ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 23 September 2014 21:03, <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Christian, >>>>>>>>>>> >>>>>>>>>>> Tried your suggestion too on 1.7.9, unfortunately it hasn't had >>>>>>>>>>> any influence even though it looked promising. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> - Ben >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tuesday, September 23, 2014 8:48:00 AM UTC+2, Christian >>>>>>>>>>> Laakmann wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi all, Luca, >>>>>>>>>>>> >>>>>>>>>>>> it seems as if you wait for synchronous responses from _all_ >>>>>>>>>>>> available Nodes, even if quorum is set to 0. In >>>>>>>>>>>> ODistributedResponseManager# >>>>>>>>>>>> collectResponse, this code handles the notification of waiting >>>>>>>>>>>> threads after all expected responses have been received. >>>>>>>>>>>> >>>>>>>>>>>> if (receivedResponses >= expectedSynchronousResponses && >>>>>>>>>>>> (!waitForLocalNode || receivedCurrentNode)) { >>>>>>>>>>>> if (completed || isMinimumQuorumReached(false)) { >>>>>>>>>>>> // NOTIFY TO THE WAITER THE RESPONSE IS COMPLETE NOW >>>>>>>>>>>> notifyWaiters(); >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> My expectation would be, that #isMinimumQuorumReached() were >>>>>>>>>>>> called as part of the first IF, i.e.: >>>>>>>>>>>> >>>>>>>>>>>> if (completed >>>>>>>>>>>> || isMinimumQuorumReached(false) >>>>>>>>>>>> || (receivedResponses >= expectedSynchronousResponses && >>>>>>>>>>>> (!waitForLocalNode || receivedCurrentNode))) { >>>>>>>>>>>> // NOTIFY TO THE WAITER THE RESPONSE IS COMPLETE NOW >>>>>>>>>>>> notifyWaiters(); >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> From my understanding, this would notify the waiting thread >>>>>>>>>>>> that pushed the task as soon as the quorum is reached or all >>>>>>>>>>>> responses from >>>>>>>>>>>> the available nodes have been collected. >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Christian >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tuesday, 23 September 2014 00:58:55 UTC+2, [email protected] >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Luca, >>>>>>>>>>>>> >>>>>>>>>>>>> Since I've been hung out to dry, I've toyed around with the >>>>>>>>>>>>> orientdb sources. I was able to establish that the bottlenecks >>>>>>>>>>>>> are indeed >>>>>>>>>>>>> sendRequest and send2Nodes. Removing sleeps in the latest 1.7.9 >>>>>>>>>>>>> release >>>>>>>>>>>>> already provided a 50% improvement (From ~1.2sec down to >>>>>>>>>>>>> ~0.55sec.). >>>>>>>>>>>>> Clearly we have different views on what is asynchronous. In your >>>>>>>>>>>>> methods, >>>>>>>>>>>>> you inarguably wait for a response from the nodes before giving a >>>>>>>>>>>>> response >>>>>>>>>>>>> to the client. IMHO asynchronous replication should update the >>>>>>>>>>>>> local >>>>>>>>>>>>> database, respond to the client and care about the rest later. I >>>>>>>>>>>>> believe >>>>>>>>>>>>> the replication methodology needs a rethink. Whilst in may be >>>>>>>>>>>>> suitable to >>>>>>>>>>>>> LAN environments, it is certainly not suitable for WAN. I still >>>>>>>>>>>>> wonder if >>>>>>>>>>>>> this is intended or even if it has ever been tested. I see >>>>>>>>>>>>> examples with >>>>>>>>>>>>> europe and usa nodes with sharding, etc.. but I can't believe >>>>>>>>>>>>> such delays >>>>>>>>>>>>> are accepted or the norm. Other technologies can do it, so should >>>>>>>>>>>>> OrientDB. >>>>>>>>>>>>> >>>>>>>>>>>>> Anyway, enough of my ramblings.... Have a good evening. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> - Ben >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> --- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "OrientDB" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> --- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "OrientDB" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "OrientDB" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OrientDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OrientDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
