It is ok for us to release 7.0.0 and implement this issue in 7.0.1. Vast majority of our applications need this functionality.
Tomas Keith Wall wrote > The test is sending persistent messages so the broker is obliged to > write them to disk. After the arrival of each message transfer, the > Broker-J awaits the sync'd to disk (after the write) before sending > the Disposition performative back to the client. The Qpid JMS Client > is awaiting the Disposition, so it is only then that the > MessageProducer#send returns and the application can send the next > message. I infer that CPP Broker must be optimistically sending the > Disposition back to the client before the data is sync'd, so that is > why you see better performance (but with a lesser guarantee). If you > were to switch the Qpid JMS Client to jms.forceAsyncSend[1], I would > expect you would see greater performance. Let me ask a higher level > question - what messaging guarantee does this application require? > > [1] https://qpid.apache.org/releases/qpid-jms-0.27.0/docs/index.html > > On 10 November 2017 at 15:50, Rob Godfrey < > rob.j.godfrey@ > > wrote: >> On 10 November 2017 at 16:39, Vavricka < > vavricka.tomas@ > > wrote: >> >>> Hi, >>> >>> hardware: >>> * Intel(R) Core(TM) i5-4570 CPU @ 3.20GHz >>> * 16 GB RAM >>> * HDD ST500DM002-1BD142 >>> >>> timings: >>> Currently Java Broker 6.1.1 seems to behave as version 7.0.0 RC. 10 - 30 >>> messages per second. Interesting is when I increase message size to >>> 10kB. >>> Messages per second are same but throughput is increased ten times. >>> When I use nonpersistent messages everything goes smooth. Thousand of >>> 1kB >>> messages are sent within 1 second. >>> >> >> So, this is more what I would expect (in the sense that 6.1 will behave >> like 7.0 - the performance is unacceptable, but I think I understand it). >> >> I *think* the issue is that we have not yet implemented the optimisations >> in the 1.0 layer for non-transactional durable messages to be processed >> asynchronously. Because of this the rate of message processing is >> dependent upon how many times fsync() can be called a second. 500/s is >> probably about right for a SAN, ~20 is what I saw from conventional hard >> drives; for SSDs with a battery backed write cache I've gotten > 1000/s. >> Because it is dependent upon the number of fsyncs rather than the >> throughput of the disk, increasing the message size will not affect the >> rate and thus you will see a linear improvement in throughput. >> >> Fixing this shouldn't actually be a huge change, and after it you should >> see something more like the C++ broker behaviour. Since this isn't (I >> think) a regression between 7.0 and 6.1 I'd suggest that we progress with >> the 7.0 release and then quickly follow that with a 7.0.1 that introduces >> the necessary optimisation (we can essentially copy over the code from >> the >> 0-x protocol layers). >> >> -- Rob >> >> >>> >>> There are no extra JVM options, just the ones which are present in >>> bin/qpid-server file. >>> >>> Heap and direct memory on broker is also default - -Xmx512m >>> -XX:MaxDirectMemorySize=1536m. I tried to increase to four times larger >>> ones >>> -Xmx2048m -XX:MaxDirectMemorySize=6000m, but there was no change in >>> messages >>> per second. >>> >>> Unfortunately vmstat gives same values pro CPU, I am sending at least >>> top >>> output. >>> >>> 6.1.1: >>> %Cpu(s): 6.9 us, 0.3 sy, 0.0 ni, 68.5 id, 24.1 wa, 0.0 hi, 0.2 si, >>> 0.0 >>> st >>> >>> 7.0.0: >>> %Cpu(s): 2.4 us, 0.4 sy, 0.0 ni, 71.2 id, 25.9 wa, 0.0 hi, 0.0 si, >>> 0.0 >>> st >>> >>> When we tried on server where message store was stored on SAN disk, >>> sending >>> of messages increased to 500 msg/sec. With C++ broker on same machine we >>> are >>> able to send 5000 msg/sec. >>> >>> ps. I cannot create queue in 7.0.0 version by webgui when queue contains >>> '.' >>> character, in 6.1.1 version queue with dot in name can be created by >>> webgui >>> >>> >>> Keith Wall wrote >>> > Hi Tomas, >>> > >>> > Nor can I reproduce any discernible difference in performance between >>> > 6.1.1 and the 7.0.0 RC with your Java code. I have not tried the C++ >>> > yet. >>> > >>> > Can you share with us: >>> > >>> > * details of the hardware (including the storage) you are using for >>> the >>> > test. >>> > * the timings you seeing for your tests for both the 6.1.1 case and >>> 7.0.0 >>> > RC >>> > * any extra JVM options you are passing either client or broker side. >>> > * size of java heap (client side) and heap and direct memory (broker) >>> > >>> > Can I suggest that you collect vmstat type information for both runs >>> > and compare? My expectation is that CPU usage, disk I/O, and network >>> > utilisation should be approximately equal between the two runs. >>> > >>> > cheers, Keith >>> >>> >>> >>> >>> >>> -- >>> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users- >>> f2158936.html >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: > users-unsubscribe@.apache >>> For additional commands, e-mail: > users-help@.apache >>> >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > users-unsubscribe@.apache > For additional commands, e-mail: > users-help@.apache -- Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
