[
https://issues.apache.org/jira/browse/ARTEMIS-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17777598#comment-17777598
]
Liviu Citu edited comment on ARTEMIS-4462 at 10/20/23 7:33 AM:
---------------------------------------------------------------
Reopened as I do not agree with the statement:
_its probable you aren't seeing an issue on 5.x broker because you aren't
currently doing anything that needs it to decode the content there. That
doesn't mean_ _the content is valid._
I have sent a message with special French character (non UTF8 encoding) and it
is accepted by 5.x broker. The content is valid and it works. I am not sure
why/when the server will have to decode something and even if it does it should
use the default charset (if I want to use UTF8 then I will specify that in the
java options or there should be a configured parameter somewhere).
Note that I am running 5.x broker in Windows with French regional settings. The
same character works also if I am using US regional settings (the encoding is
the same ISO-88591).
If Artemis is always expecting messages to be UTF8 encoded then this should be
clearly stipulated in the official documentation because right now there is no
reference for it. Actually there is no encoding reference at all in the
documentation.
I am also for using UTF8 but right now calling an extra function on all our
applications for UTF8 encoding is a change in functionality which might have
performance cost.
Perhaps a message property might need to be decoded on the server side and has
to be UTF8 which I can agree but not the message text (which can be a very
large string and encoding/decoding takes CPU time).
was (Author: JIRAUSER300236):
Reopened as I do not agree with the statement:
_its probable you aren't seeing an issue on 5.x broker because you aren't
currently doing anything that needs it to decode the content there. That
doesn't mean_ _the content is valid._
I have sent a message with special French character (non UTF8 encoding) and it
is accepted by 5.x broker. The content is valid and it works. I am not sure
why/when the server will have to decode something and even if it does it should
use the default charset (if I want to use UTF8 then I will specify that in the
java options or there should be a configured parameter somewhere).
Note that I am running 5.x broker in Windows with French regional settings. The
same character works also if I am using US regional settings (the encoding is
the same ISO-88591).
If Artemis is always expecting messages to be UTF8 encoded then this should be
clearly stipulated in the official documentation because right now there is no
reference for it. Actually there is no encoding reference at all in the
documentation.
I am also for using UTF8 but right now calling an extra function on all our
applications for UTF8 encoding is a change in functionality which might have
performance cost.
Perhaps a message property might need to be decoded which I can agree but not
the message text (can be a very large string and encoding/decoding takes CPU
time).
> Non-UTF-8 messages containing special characters are not supported
> ------------------------------------------------------------------
>
> Key: ARTEMIS-4462
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4462
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.30.0
> Reporter: Liviu Citu
> Priority: Major
>
> When a text message non-UTF-8 (ISO-8859-15) containing special characters is
> sent from ActiveMQ CPP client to Artemis, an exception
> "java.io.UTFDataFormatException" is seen in Artemis server log.
> Although the exception is thrown as a warning, the message gets rejected by
> the server.
> Below the Artemis server log:
>
> {noformat}
> 2023-09-29 11:34:32,736 WARN
> [org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection]
> Errors occurred during the buffering operation
>
> java.io.UTFDataFormatException: null
> at
> org.apache.activemq.util.MarshallingSupport.convertUTF8WithBuf(MarshallingSupport.java:386)
> ~[activemq-client-5.17.2.jar:5.17.2]
> at
> org.apache.activemq.util.MarshallingSupport.readUTF8(MarshallingSupport.java:358)
> ~[activemq-client-5.17.2.jar:5.17.2]
> at
> org.apache.activemq.artemis.core.protocol.openwire.OpenWireMessageConverter.writeTextType(OpenWireMessageConverter.java:233)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.artemis.core.protocol.openwire.OpenWireMessageConverter.inbound(OpenWireMessageConverter.java:128)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.artemis.core.protocol.openwire.amq.AMQSession.send(AMQSession.java:376)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection$CommandProcessor.processMessage(OpenWireConnection.java:1671)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:769)
> ~[activemq-client-5.17.2.jar:5.17.2]
> at
> org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection.act(OpenWireConnection.java:369)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.artemis.utils.actors.ThresholdActor.doTask(ThresholdActor.java:68)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
> ~[artemis-commons-2.30.0.jar:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> [?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> [?:?]
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [artemis-commons-2.30.0.jar:?]
> 2023-09-29 11:34:32,753 WARN
> [org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection]
> Errors occurred during the buffering operation
> org.apache.activemq.artemis.api.core.ActiveMQException: null
> at
> org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection$CommandProcessor.processMessage(OpenWireConnection.java:1674)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:769)
> ~[activemq-client-5.17.2.jar:5.17.2]
> at
> org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection.act(OpenWireConnection.java:369)
> ~[artemis-openwire-protocol-2.30.0.jar:2.30.0]
> at
> org.apache.activemq.artemis.utils.actors.ThresholdActor.doTask(ThresholdActor.java:68)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
> ~[artemis-commons-2.30.0.jar:?]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
> ~[artemis-commons-2.30.0.jar:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> [?:?]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> [?:?]
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [artemis-commons-2.30.0.jar:?]
> 2023-09-29 11:34:32,755 WARN
> [org.apache.activemq.artemis.core.protocol.openwire.OpenWireConnection]
> Errors occurred during the buffering operation
> java.io.UTFDataFormatException: null
>
> {noformat}
>
> Just create a small test program using ActiveMQ CPP and send a message to
> Artemis containing a french or german specific character.
>
> I have found an old discussion from ActiveMQ Community but it was focused on
> ActiveMQ CPP rather on Artemis.
> [https://lists.apache.org/thread/vywv1gk32mkhvj7sghnvlf7ng5zb1obp]
>
> This is a regression compared with ActiveMQ Classic which does not have this
> issue.
> Does Artemis support non-UTF-8 encoding? Do we need now to encode all
> messages to UTF-8 before sending them to the server? Similarly to decode them
> upon receiving?
> This will cause some problems on our end because:
> * we have many clients already using the ActiveMQ classic and migrating them
> to Artemis will cause issues when migrating KahaDB to Artemis because all the
> messages have to be encoded to UTF-8 otherwise they will not work in Artemis
> * encoding/decoding every message will impact the overall performance. We
> have applications handling thousands of messages every day and conversion of
> these messages will increase the time spent during communication with JMS
> broker
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
