Sending a message this large will make it to be converted as a large
stream and saved as a file on the server anyways...
The copy between ByteBuffer and a byte array would become irrelevant
at this point.
If you want to send a large message this big, you could save it to a
file, and pass it over the property that would stream the file
directly. (Available on core only)
And don't convert a large message between protocols.. use a single
protocol of your choice.
On Thu, Mar 25, 2021 at 3:44 AM <tobias.w...@t-systems.com> wrote:
>
> What are you talking about?
> I want to send byte buffers with the size up to 500MB to a queue!
> What do you mean with string and micro optimization!
> The core api fails in my test already with sending/receiving a simple string,
> yes, a simple test!
> Does it make sense to have a 500mb native netty buffer received via wire and
> tunnel it back through the jvm?
> I don`t think so!
> Does it make sense to tunnel megabytes of data through a stream or even an
> byte array through the jvm?
> Where ist the advantage of jms?
> Why shall I not use the existing core api functions when the already support
> a netty buffer?
>
> -----Ursprüngliche Nachricht-----
> Von: Justin Bertram <jbert...@apache.org>
> Gesendet: Mittwoch, 24. März 2021 17:56
> An: users@activemq.apache.org
> Betreff: Re: send message
>
> Well said, Tim. I was going to say the same thing about premature
> optimization.
>
>
> Justin
>
> On Wed, Mar 24, 2021 at 7:04 AM Tim Bain <tb...@alumni.duke.edu> wrote:
>
> > One thing to keep in mind is that although the code may be slightly
> > slower, if it lets you use an API that is more favorable (which might
> > mean better documented, more portable, more stable across future
> > versions, easier for others to maintain because they're already
> > familiar with it, or a number of other potential advantages), that might be
> > a win.
> >
> > Only you know your use case and whether the performance advantages of
> > what you're trying to do actually outweigh the types of advantages I
> > referenced in the first paragraph. But remember that premature
> > optimization is the root of all evil, so make sure that your
> > micro-optimization of avoiding the creation of a String is really a
> > significant enough optimization in the context of your use case to
> > justify the disadvantages of what you're trying to do.
> >
> > Tim
> >
> > On Tue, Mar 23, 2021, 9:20 AM Clebert Suconic
> > <clebert.suco...@gmail.com>
> > wrote:
> >
> > > Not on the API itself.
> > >
> > > Try the core api example I sent you... and let me know what is
> > > different from yours.
> > >
> > > On Tue, Mar 23, 2021 at 10:34 AM <tobias.w...@t-systems.com> wrote:
> > > >
> > > > I want to use the jms layer, but I've a netty application and I
> > > > want to
> > > write the ByteBuf mostly efficent to the queue.
> > > > I think it makes no sense to convert it back from a native buffer
> > > > into
> > a
> > > byte array or stream and tunnel this throught the jvm!
> > > >
> > > > Is there a way to use a ByteBuf with jms?
> > > >
> > > > -----Ursprüngliche Nachricht-----
> > > > Von: Clebert Suconic <clebert.suco...@gmail.com>
> > > > Gesendet: Dienstag, 23. März 2021 15:00
> > > > An: users@activemq.apache.org
> > > > Betreff: Re: send message
> > > >
> > > > You were supposed to call reset right before sending though... but
> > > > I
> > did
> > > not need it on my test.. I will post a link...
> > > >
> > > > Anyway, if you use the JMS API, the JMS layer is very thin, and I
> > > > don't
> > > think it would affect your performance in any way. you could create
> > > a BytesMessage and all the detail would be hidden away from the
> > > internal
> > API.
> > > >
> > > > On Tue, Mar 23, 2021 at 9:57 AM Clebert Suconic <
> > > clebert.suco...@gmail.com> wrote:
> > > > >
> > > > > I'm adding a test doing exactly what you did, and it's working...
> > > > >
> > > > > give me 10 minutes and I will post a link here...
> > > > >
> > > > > On Tue, Mar 23, 2021 at 9:56 AM <tobias.w...@t-systems.com> wrote:
> > > > > >
> > > > > > Same Issue!
> > > > > > public void sendMessageToSmtpQueue(String text) throws
> > > ActiveMQException {
> > > > > > ClientSession session =
> > > this.sessionFactory.createSession();
> > > > > > try {
> > > > > > session.start();
> > > > > > ClientMessage message =
> > > session.createMessage(true);
> > > > > > message.setType(ClientMessage.TEXT_TYPE);
> > > > > > message.getBodyBuffer().resetReaderIndex();
> > > > > > message.getBodyBuffer().writeString(text);
> > > > > > ClientProducer producer =
> > > session.createProducer(ACTIVE_MQ_SMTP_QUEUE);
> > > > > > producer.send(message);
> > > > > > } finally {
> > > > > > session.close();
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > java.lang.IndexOutOfBoundsException: Error reading in
> > > > > > simpleString,
> > > length=1953068645 is greater than readableBytes=3
> > > > > > at
> > >
> > org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(Sim
> > pleString.java:185)
> > > > > > at
> > >
> > org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(Sim
> > pleString.java:173)
> > > > > > at
> > >
> > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.rea
> > dStringInternal(ChannelBufferWrapper.java:113)
> > > > > > at
> > >
> > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.rea
> > dString(ChannelBufferWrapper.java:98)
> > > > > > at com.tsystems.gematik.kim.mailserver.mq
> > >
> > .MailServerActiveMQClient.receiveTextMessageFromSmtpQueue(MailServerAc
> > tiveMQClient.java:94)
> > > > > > at com.tsystems.gematik.kim.mailserver.mq
> > >
> > .MailServerActiveMQClientTest.sendAndReceiveTextMessage(MailServerActi
> > veMQClientTest.java:37)
> > > > > > at
> > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Nati
> > > ve
> > > Method)
> > > > > > at
> > >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodAccessorImpl.java:62)
> > > > > > at
> > >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatingMethodAccessorImpl.java:43)
> > > > > > at
> > java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > > > > > at
> > >
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkM
> > ethod.java:50)
> > > > > > at
> > >
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall
> > able.java:12)
> > > > > > at
> > >
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMet
> > hod.java:47)
> > > > > > at
> > >
> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:17)
> > > > > > at
> > > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > > > > > at
> > >
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> > er.java:78)
> > > > > > at
> > >
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> > er.java:57)
> > > > > > at
> > > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > > > > > at
> > > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> > > > > > at
> > > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> > > > > > at
> > > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> > > > > > at
> > > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> > > > > > at
> > org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > > > > > at
> > >
> > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4T
> > estReference.java:89)
> > > > > > at
> > >
> > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.
> > java:41)
> > > > > > at
> > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(Remote
> > TestRunner.java:541)
> > > > > > at
> > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(Remote
> > TestRunner.java:763)
> > > > > > at
> > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestR
> > unner.java:463)
> > > > > > at
> > > > > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTe
> > > > > > stRunner.java:209)
> > > > > >
> > > > > >
> > > > > > -----Ursprüngliche Nachricht-----
> > > > > > Von: Clebert Suconic <clebert.suco...@gmail.com>
> > > > > > Gesendet: Dienstag, 23. März 2021 13:48
> > > > > > An: users@activemq.apache.org
> > > > > > Betreff: Re: send message
> > > > > >
> > > > > > Can you try calling
> > > > > > message.getBodyBuffer().resetReaderIndex();
> > > before the send?
> > > > > >
> > > > > > if that does not work I will try your code and see what happens.
> > > > > > (Let me know if doesn't please)
> > > > > >
> > > > > > On Tue, Mar 23, 2021 at 7:55 AM Gary Tully
> > > > > > <gary.tu...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > maybe take inspiration from
> > > > > > >
> > https://github.com/apache/activemq-artemis/blob/master/tests/integ
> > > > > > > rati
> > > > > > >
> > on-tests/src/test/java/org/apache/activemq/artemis/tests/integrati
> > > > > > > on/c lient/MessageBufferTest.java that passes a string, but
> > > > > > > there are corresponding byte[] or buffer variants in the
> > > > > > > api. If you want to skip the decoding, to access the raw
> > > > > > > buffer, you need to parse the type to get to the appropriate part
> > > > > > > of the buffer.
> > > > > > > There are loads of usage examples in the tests and they all
> > > > > > > work, start there and break it as you go.
> > > > > > >
> > > > > > > the issue is the wire level encoding that the client does,
> > > > > > > the
> > JMS
> > > > > > > api hides this, as do the typed accessors, but you can get
> > > > > > > direct access to the encoded buffers via the core api as you
> > > > > > > are doing but you need to be type aware, for example a
> > > > > > > string can be utf-8 encoded or it can be raw bytes on the
> > > > > > > wire, depending on the size and chars, the encoding handles
> > > > > > > and hides this detail. But if you go for direct access, you need
> > > > > > > to be aware of the encoding.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, 23 Mar 2021 at 07:37, <tobias.w...@t-systems.com> wrote:
> > > > > > > >
> > > > > > > > getDataBuffer gives same result! Already tried!
> > > > > > > >
> > > > > > > > -----Ursprüngliche Nachricht-----
> > > > > > > > Von: Clebert Suconic <clebert.suco...@gmail.com>
> > > > > > > > Gesendet: Dienstag, 23. März 2021 03:19
> > > > > > > > An: users@activemq.apache.org
> > > > > > > > Betreff: Re: send message
> > > > > > > >
> > > > > > > > Why you don't use the JMS API for this?
> > > > > > > >
> > > > > > > > or if you really want to use the core API, use the
> > > getReadOnlyBuffer() or getDataBuffer() on the Message instead.
> > > > > > > >
> > > > > > > > On Mon, Mar 22, 2021 at 12:01 PM Justin Bertram <
> > > jbert...@apache.org> wrote:
> > > > > > > > >
> > > > > > > > > The stack-trace indicates you're invoking the "readString"
> > > > > > > > > method on line
> > > > > > > > > 93 of MailServerActiveMQClient.java, but I don't see
> > > > > > > > > that
> > call
> > > > > > > > > in the code you pasted. Therefore, that stack-trace
> > > > > > > > > doesn't seem correct. Can you clarify this?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Justin
> > > > > > > > >
> > > > > > > > > On Mon, Mar 22, 2021 at 10:55 AM Dondorp, Erwin
> > > > > > > > > <erwin.dond...@cgi.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Fyi:
> > > > > > > > > > 1953068645(decimal) = 74697665(hexadecimal) =
> > > > > > > > > > "tive"(ascii
> > > > > > > > > > text) And "tive" is likely part of the string "ActiveMQ"?
> > > > > > > > > > e.
> > > > > > > > > >
> > > > > > > > > > -----Oorspronkelijk bericht-----
> > > > > > > > > > Van: tobias.w...@t-systems.com
> > > > > > > > > > <tobias.w...@t-systems.com>
> > > > > > > > > > Verzonden: maandag 22 maart 2021 16:41
> > > > > > > > > > Aan: users@activemq.apache.org
> > > > > > > > > > Onderwerp: AW: send message
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > EXTERNAL SENDER: Do not click any links or open any
> > > attachments unless
> > > > > > > > > > you trust the sender and know the content is safe.
> > > > > > > > > > EXPÉDITEUR EXTERNE: Ne cliquez sur aucun lien et
> > n’ouvrez
> > > aucune pièce
> > > > > > > > > > jointe à moins qu’ils ne proviennent d’un expéditeur
> > fiable,
> > > > > > > > > > ou que vous ayez l'assurance que le contenu provient
> > > > > > > > > > d'une
> > > source sûre.
> > > > > > > > > >
> > > > > > > > > > While sending / receiving a text message I get this
> > > > > > > > > >
> > > > > > > > > > java.lang.IndexOutOfBoundsException: Error reading in
> > > > > > > > > > simpleString,
> > > > > > > > > > length=1953068645 is greater than readableBytes=3
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(Sim
> > pleString.java:185)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.apache.activemq.artemis.api.core.SimpleString.readSimpleString(Sim
> > pleString.java:173)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.rea
> > dStringInternal(ChannelBufferWrapper.java:113)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.rea
> > dString(ChannelBufferWrapper.java:98)
> > > > > > > > > > at com.tsystems.gematik.kim.mailserver.mq
> > > > > > > > > >
> > >
> > .MailServerActiveMQClient.receiveTextMessageFromSmtpQueue(MailServerAc
> > tiveMQClient.java:93)
> > > > > > > > > > at com.tsystems.gematik.kim.mailserver.mq
> > > > > > > > > >
> > >
> > .MailServerActiveMQClientTest.sendAndReceive(MailServerActiveMQClientT
> > est.java:28)
> > > > > > > > > > at
> > > > > > > > > >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invo
> > > > > > > > > > ke0(
> > > > > > > > > > Nati
> > > > > > > > > > ve
> > > > > > > > > > Method)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeM
> > ethodAccessorImpl.java:62)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Del
> > egatingMethodAccessorImpl.java:43)
> > > > > > > > > > at
> > > java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkM
> > ethod.java:50)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall
> > able.java:12)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMet
> > hod.java:47)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:17)
> > > > > > > > > > at
> > > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> > er.java:78)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> > er.java:57)
> > > > > > > > > > at
> > > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > > > > > > > > > at
> > > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> > > > > > > > > > at
> > > > > > > > > >
> > > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> > > > > > > > > > at
> > > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> > > > > > > > > > at
> > > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> > > > > > > > > > at
> > > org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4T
> > estReference.java:89)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.
> > java:41)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(Remote
> > TestRunner.java:541)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(Remote
> > TestRunner.java:763)
> > > > > > > > > > at
> > > > > > > > > >
> > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestR
> > unner.java:463)
> > > > > > > > > > at
> > > > > > > > > >
> > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
> > > > > > > > > > Remo
> > > > > > > > > > teTe
> > > > > > > > > > stRunner.java:209)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > -----Ursprüngliche Nachricht-----
> > > > > > > > > > Von: Justin Bertram <jbert...@apache.org>
> > > > > > > > > > Gesendet: Montag, 22. März 2021 16:34
> > > > > > > > > > An: users@activemq.apache.org
> > > > > > > > > > Betreff: Re: send message
> > > > > > > > > >
> > > > > > > > > > What actually fails? Do you have a stack-trace?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Justin
> > > > > > > > > >
> > > > > > > > > > On Mon, Mar 22, 2021 at 9:53 AM
> > > > > > > > > > <tobias.w...@t-systems.com
> > >
> > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I try to send and receive a netty ByteBuf message,
> > > > > > > > > > > but it
> > > fails.
> > > > > > > > > > > What I'm doing wrong here? I even don't know ist he
> > > > > > > > > > > mistake in sending or receiving!
> > > > > > > > > > > I was thinking to use the jms layer, but I'm
> > > > > > > > > > > receiving
> > the
> > > > > > > > > > > data buffer throught a netty buffer and I want to
> > > > > > > > > > > avoid
> > to
> > > > > > > > > > > convert the buffer to a byte array!
> > > > > > > > > > >
> > > > > > > > > > > public void sendMessageToSmtpQueue(ByteBuf
> > > > > > > > > > > buf) throws ActiveMQException {
> > > > > > > > > > > ClientSession session =
> > > this.sessionFactory.createSession();
> > > > > > > > > > > try {
> > > > > > > > > > > session.start();
> > > > > > > > > > > ClientMessage message =
> > > session.createMessage(true);
> > > > > > > > > > >
> > > > > > > > > > > message.getBodyBuffer().writeBytes(buf, 0,
> > > buf.readableBytes());
> > > > > > > > > > > ClientProducer producer =
> > > > > > > > > > > session.createProducer(ACTIVE_MQ_SMTP_QUEUE);
> > > > > > > > > > > producer.send(message);
> > > > > > > > > > > } finally {
> > > > > > > > > > > session.close();
> > > > > > > > > > > }
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > public ActiveMQBuffer
> > receiveMessageFromSmtpQueue()
> > > > > > > > > > > throws ActiveMQException {
> > > > > > > > > > > ActiveMQBuffer result;
> > > > > > > > > > > ClientSession session =
> > > this.sessionFactory.createSession();
> > > > > > > > > > > try {
> > > > > > > > > > > session.start();
> > > > > > > > > > > ClientConsumer consumer =
> > > > > > > > > > > session.createConsumer(ACTIVE_MQ_SMTP_QUEUE);
> > > > > > > > > > > ClientMessage message =
> > > consumer.receive();
> > > > > > > > > > > result =
> > > > > > > > > > > ActiveMQBuffers.fixedBuffer(message.getBodyBufferSiz
> > > > > > > > > > > e());
> > > > > > > > > > >
> > > message.getBodyBuffer().readBytes(result);
> > > > > > > > > > > } finally {
> > > > > > > > > > > session.close();
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > return result;
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > public void startServer() throws Exception {
> > > > > > > > > > > this.configuration = new
> > ConfigurationImpl();
> > > > > > > > > > >
> > > > > > > > > > > this.configuration.addAcceptorConfiguration("in-vm",
> > > > > > > > > > > ACTIVE_MQ_EMBEDDED_SERVER_URL);
> > > > > > > > > > >
> > > this.configuration.setPersistenceEnabled(true);
> > > > > > > > > > >
> > this.configuration.setSecurityEnabled(false);
> > > > > > > > > > >
> > > > > > > > > > > this.configuration.addQueueConfiguration(new
> > > > > > > > > > > QueueConfiguration(ACTIVE_MQ_SMTP_QUEUE));
> > > > > > > > > > >
> > > > > > > > > > > this.server = new
> > > ActiveMQServerImpl(this.configuration);
> > > > > > > > > > > this.server.start();
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Clebert Suconic
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Clebert Suconic
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Clebert Suconic
> > > >
> > > >
> > > >
> > > > --
> > > > Clebert Suconic
> > >
> > >
> > >
> > > --
> > > Clebert Suconic
> > >
> >
--
Clebert Suconic
