Hi Martyn,
I've just opened a JIRA related to this issue: it's the tkt ARTEMIS-917
(https://issues.apache.org/jira/browse/ARTEMIS-917). Of course it's currently
unassigned: let me know if I have to assign it to someone in particular.
A Thing That may help you (even if I understand you've already figured out
where the problem lies): a developer of ours did put the Artemis code under
debug and he has noticed that the message is yes, written correctly on the file
system, but then it is read in a wrong way. It seems a matter of pointer
position, which is kept to the end of metadata instead of the end of message
body while reading it. The involved classes should be the followings:
AbstractJournalStorageManager.java
--> this is the class that probably doesn't correctly values the pointer.
MessageImpl.java
--> this (if I understand well) is the class containing the "decode" method
used to read the message. A question: do you think the last bold line, below,
could be a solution?
public void decode(final ActiveMQBuffer buff) {
int start = buff.readerIndex();
endOfBodyPosition = buff.readInt();
endOfMessagePosition = buff.getInt(endOfBodyPosition -
BUFFER_HEADER_SPACE + start);
int length = endOfMessagePosition - BUFFER_HEADER_SPACE;
buffer.setIndex(0, BUFFER_HEADER_SPACE);
buffer.writeBytes(buff, start, length);
decode();
buff.readerIndex(start + length);
buff.writerIndex(start + endOfBodyPosition);
}
Thanks a lot for your help and your patience.
Francesco
________________________________
From: Martyn Taylor <mtay...@redhat.com>
Sent: Wednesday, January 11, 2017 3:53:00 PM
To: users@activemq.apache.org
Subject: Re: MQTT retained messages with weird characters
It looks to me like the full message buffer (not just the body) is getting
returned to the client. Clebert I think you're right in that we do some
hackery to handle retained messages. I suspect this is where the error is.
Francesco could you create a JIRA with some reproducer steps and I'll aim
to take a look.
Thanks
On Wed, Jan 11, 2017 at 2:03 PM, Clebert Suconic <clebert.suco...@gmail.com>
wrote:
> the MQTT ProtocolManager does some playing with retaining, perhaps
> there's something wrong with reloading from journal on retaining. I
> was wondering if you could send us a test showing the issue with MQTT
> so we could see if there's anything specific to your test while using
> MQTT.
>
> On Wed, Jan 11, 2017 at 4:22 AM, Francesco PADOVANI
> <francesco.padov...@bticino.it> wrote:
> > Unfortunately no (not yet).
> >
> > Anyway, it's a clean Artemis installation of version 1.5.1, made by
> > following the user manual instructions at
> > https://activemq.apache.org/artemis/docs/1.5.1/using-server.html.
> >
> > The installation was done by using a dedicated user and also the artemis
> > java process runs with this dedicated user. Attached you can find my
> > broker.xml configuration file: it's pretty much the same default created
> > during the installation procedure, but for the acceptors (which I've
> > customized for my MQTT purpose) and the addition of parameter
> > "<last-value-queue>true</last-value-queue>" inside the address-setting
> > section (but I tried with and without it and my issue persists).
> >
> > The only other configuration I changed is the heap size dedicated to the
> > process, by setting "-Xms4096M -Xmx4096M" among the JAVA_ARGS in the
> > artemis.profile file.
> >
> >
> > The platform where Artemis broker is installed is:
> >
> >
> > - Machine:
> >
> > It's an EC2 instance on AWS cloud of type m4.large: 2 vCPU, 8G RAM and
> SSD
> > storage.
> >
> >
> > - OS:
> >
> > CentOS Linux release 7.3.1611 (Core) - x86_64
> >
> >
> > - JVM:
> >
> > java version "1.8.0_111"
> > Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
> >
> >
> > - Distro package version of libaio:
> >
> > Arch : x86_64
> > Version : 0.3.109
> > Release : 13.el7
> >
> >
> > It seems like the problem relates the Persistence (File Journal?). Until
> the
> > broker keeps the retained messages in ram it works. When it needs to
> recover
> > the retained messages from the file system, clients receive them with
> weird
> > chars.
> > And I suspect the same issue appears when RAM is full and messages start
> to
> > be paged... but this case is a bit more difficult to test in a systematic
> > way.
> >
> >
> > Francesco
> >
> >
> >
> > ________________________________
> > From: Clebert Suconic <clebert.suco...@gmail.com>
> > Sent: Tuesday, January 10, 2017 7:43 PM
> > To: users@activemq.apache.org
> > Subject: Re: MQTT retained messages with weird characters
> >
> > Do you have a working test you can share?
> >
> > On Tue, Jan 10, 2017 at 12:07 PM, Francesco PADOVANI
> > <francesco.padov...@bticino.it> wrote:
> >> Hello,
> >>
> >> I'm trying the Apache Artemis Broker (ver. 1.5.1) for MQTT protocol.
> >>
> >> Exactly I'm testing the retained feature for messages of MQTT protocol.
> >>
> >> While the broker is up it seems to work well:
> >>
> >> a) a client publishes a retained message to a specific topic
> >>
> >> b) any client which then subscribes that topic receives the retained
> >> message (any time it re-subscribes the topic it receives the last
> retained
> >> message ...perfect!)
> >>
> >> But it happens a strange thing when I restart the broker. And I mean I
> >> restart the broker without any change on configuration: simply I make a
> >> "artemis-service stop" and "artemis-service start". After The broker is
> up
> >> and running, if a client subscribes the previous topic it still
> receives the
> >> retained message, but with weird characters appended to it. For new
> retained
> >> messages published it work well again... but the previous ones (before
> the
> >> broker restart) are sent all with these weird characters. E.g., the
> >> following is a retained message published before the broker restart:
> >>
> >> "test 25 for a retained message 20170110"
> >>
> >> And this is exactly how clients get it before the broker restart.
> >>
> >> Instead, after broker restart, the same retained message is received by
> >> clients in the following way:
> >>
> >> test 25 for a retained message 20170110?
> >> ? 6$sys.mqtt.retain..cro.test5 ?
> >> &mqtt.message.retain ? mqtt.qos.level
> >>
> >>
> >> Is this my bad configuration (but I don't know ehere)? Or anything else?
> >> Or what?
> >>
> >> Someone can help me?
> >>
> >>
> >> Thanks in advance.
> >>
> >>
> >> Francesco
> >>
> >> ________________________________
> >>
> >> Ce message, ainsi que tous les fichiers joints à ce message, peuvent
> >> contenir des informations sensibles et/ ou confidentielles ne devant pas
> >> être divulguées. Si vous n'êtes pas le destinataire de ce message (ou
> que
> >> vous recevez ce message par erreur), nous vous remercions de le notifier
> >> immédiatement à son expéditeur, et de détruire ce message. Toute copie,
> >> divulgation, modification, utilisation ou diffusion, non autorisée,
> directe
> >> ou indirecte, de tout ou partie de ce message, est strictement
> interdite.
> >>
> >>
> >> This e-mail, and any document attached hereby, may contain confidential
> >> and/or privileged information. If you are not the intended recipient (or
> >> have received this e-mail in error) please notify the sender
> immediately and
> >> destroy this e-mail. Any unauthorized, direct or indirect, copying,
> >> disclosure, distribution or other use of the material or parts thereof
> is
> >> strictly forbidden.
> >
> >
> >
> > --
> > Clebert Suconic
> >
> > ________________________________
> >
> > Ce message, ainsi que tous les fichiers joints à ce message, peuvent
> > contenir des informations sensibles et/ ou confidentielles ne devant pas
> > être divulguées. Si vous n'êtes pas le destinataire de ce message (ou que
> > vous recevez ce message par erreur), nous vous remercions de le notifier
> > immédiatement à son expéditeur, et de détruire ce message. Toute copie,
> > divulgation, modification, utilisation ou diffusion, non autorisée,
> directe
> > ou indirecte, de tout ou partie de ce message, est strictement interdite.
> >
> >
> > This e-mail, and any document attached hereby, may contain confidential
> > and/or privileged information. If you are not the intended recipient (or
> > have received this e-mail in error) please notify the sender immediately
> and
> > destroy this e-mail. Any unauthorized, direct or indirect, copying,
> > disclosure, distribution or other use of the material or parts thereof is
> > strictly forbidden.
>
>
>
> --
> Clebert Suconic
>
________________________________
Ce message, ainsi que tous les fichiers joints à ce message, peuvent contenir
des informations sensibles et/ ou confidentielles ne devant pas être
divulguées. Si vous n'êtes pas le destinataire de ce message (ou que vous
recevez ce message par erreur), nous vous remercions de le notifier
immédiatement à son expéditeur, et de détruire ce message. Toute copie,
divulgation, modification, utilisation ou diffusion, non autorisée, directe ou
indirecte, de tout ou partie de ce message, est strictement interdite.
This e-mail, and any document attached hereby, may contain confidential and/or
privileged information. If you are not the intended recipient (or have received
this e-mail in error) please notify the sender immediately and destroy this
e-mail. Any unauthorized, direct or indirect, copying, disclosure, distribution
or other use of the material or parts thereof is strictly forbidden.
