@Otavia I run the test and this also happens on Camel 3.19.0 (JDK11). The
result:
Caused by: java.nio.charset.UnmappableCharacterException: Input length = 1
I am not expert on charsets, but when a character is not available /
unmappable I would expect that it is replaced by for example a '?'.
Obviously this
does not happen. Not directly a Camel issue, but more a Java
(java.nio.charset) issue.
Some suggestions:
1. Replace the character to mappable character. Something like
.transform(simple( "${body.replace('&', '&')}"))
2. Use convertBody. Something like this:
<route id="charset_test">
<from
uri="file:./input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
<log message="PMA : processing ${file:name} (length
${file:length})..."/>
<convertBodyTo charset="ISO-8859-1" type="java.lang.String"/>
<to uri="file:./output"/>
</route>
3. Write a processor
Maybe avoiding "java.nio.charset" by using a processor:
.process(exchange -> {
String encodedWithUTF8 = exchange.getIn().getBody(String.class);
String decodedToISO88591 = new
String(encodedWithUTF8.getBytes("UTF-8"), "ISO-8859-1");
exchange.getIn().setBody(decodedToISO88591);
}
Raymond
On Thu, Nov 3, 2022 at 4:00 PM Ephemeris Lappis <ephemeris.lap...@gmail.com>
wrote:
> Hello again.
>
> Well, as I said in my previous mail, I'm not sure that Camel is the
> single component that's involved in our issues.
> I've built a very little java project with a very simple route that
> does the same operation as my blueprint.
>
> I've got strange results.
>
> When running with Camel 2.17.0 (the version that fails in our
> production system), It works (yes, very strange) !
> But when I try switching to 2.25.4, it fails again.
>
> In both cases I only use Camel and the logger over a Java 1.8 runtime.
>
> I attach the zipped project. Perhaps someone can test it too.
>
> This let me think that "something" elsewhere is involved that makes
> similar routes work or fail, and, if I'm not mistaken, perhaps we can
> control this "something"...
>
> Any idea from experts ???
>
> Thanks again.
>
> Regards.
>
>
>
>
> Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> <angusyo...@gmail.com> a écrit :
> >
> > Hi,
> >
> > I wouldn't know about the reason for this behavior, but my suggestion is
> to
> > try with a newer version of Camel. Camel version 2.x is not maintained
> > anymore.
> >
> > Please, can you try with a newer version of Camel? Something like 3.18.3
> > ... and let us know if this is still a problem?
> >
> > Kind regards
> >
> > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> ephemeris.lap...@gmail.com>
> > wrote:
> >
> > > Hello.
> > >
> > > The issue happens on different versions of Camel with different
> > > configurations :
> > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > >
> > > The last one is the production environment where the issue is
> critical...
> > >
> > > What's strange, is that the same little blueprint works as expected
> > > without any error with :
> > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > >
> > > Any idea of the origin of the issue, and if a workaround exists : an
> > > endpoint option, some configuration, anything ?
> > >
> > > Thanks.
> > >
> > > Regards.
> > >
> > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > <angusyo...@gmail.com> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > What version of Camel are you using?
> > > >
> > > > Kind regards
> > > >
> > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > ephemeris.lap...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello again.
> > > > >
> > > > > Sorry to answer my own question...
> > > > >
> > > > > I've read again the file component documentation looking for any
> > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > example that it's given for using charset :
> > > > >
> > > > > The charset option allows for configuring an encoding of the files
> on
> > > > > both the consumer and producer endpoints. For example if you read
> > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> do:
> > > > >
> > > > > from("file:inbox?charset=utf-8")
> > > > > .to("file:outbox?charset=iso-8859-1")
> > > > >
> > > > > And this seems to be always failing with the
> > > > > UnmappableCharacterException when a character from the input has no
> > > > > mapped character in the output charset...
> > > > >
> > > > > Nobody has reported it yet ?
> > > > >
> > > > > Thanks for your feedback.
> > > > >
> > > > > Regards.
> > > > >
> > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > <ephemeris.lap...@gmail.com> a écrit :
> > > > > >
> > > > > > Hello.
> > > > > >
> > > > > > I've been using the File component for years, and discover today
> a
> > > > > > strange exception when using a charset on a producer endpoint.
> > > > > >
> > > > > > Example :
> > > > > >
> > > > > > <route id="pma-file">
> > > > > > <from
> > > > >
> > >
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > <log message="PMA : processing ${file:name} (length
> > > ${file:length})..."/>
> > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > </route>
> > > > > >
> > > > > > If I put a file that contains only characters that exist in
> > > > > > ISO-8859-1, no problem.
> > > > > > If the file contains an € (euro) for example, an exception is
> thrown
> > > > > > (see at the end of the mail) :
> > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > >
> > > > > > A trivial java program, with the same jvm on the same machine,
> that
> > > > > > reads a file in UTF-8 and writes its content into another file
> with
> > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > >
> > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > missing
> > > > > > option on the endpoint ?
> > > > > > I've not found any option for that...
> > > > > >
> > > > > > Thanks for your help.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > The exception :
> > > > > >
> org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > length =
> > > > > 1
> > > > > > at
> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > ~[?:1.8.0_242]
> > > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > ... 22 more
> > > > > > Suppressed:
> java.nio.charset.UnmappableCharacterException:
> > > > > > Input length = 1
> > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748)
> > > [?:1.8.0_242]
> > > > > > 1
> > > > >
> > > >
> > > >
> > > > --
> > > > Otavio R. Piske
> > > > http://orpiske.net
> > >
> >
> >
> > --
> > Otavio R. Piske
> > http://orpiske.net
>
