Hi, I have had a look into the source code:
If you look into the StreamCacheConverter an InputSteam will create a StreamCache object by copying the data to a CachedOutputStream and then calling the newStreamCache() method. That will generate a FileInputStreamCache or a ByteArrayStreamCache. Both classes are StreamCache and InputStream at the same time, so if the StreamCache is converted to an InputStream the instance itself is returned. In the parallel multicast case (and probably also in the splitter), having a StreamCache before the split will result in all branches getting the same InputStream instance (which will lead to a pretty garbled output if two threads are reading from it at the same time). I see two potential ways how to resolve this issue: One way would be to change the StreamCache instances to not implement InputStream, but to provide a type converter to InputStream which will generate a new InputSteam for each consumer. The other way would be copy the StreamCache instance in case of a multicast. What do you think would be the better solution? Best regards Stephan -----Original Message----- From: lakshmi.prashant [mailto:lakshmi.prash...@gmail.com] Sent: Mittwoch, 14. Januar 2015 12:43 To: users@camel.apache.org Subject: Data Corruption in SFTP in Parallel Multicast branches Hi, • In case if there are SFTP receivers at the end of the branch(es) of Parallel Multicast, the payload gets corrupted in 1 or more branches. o Any 1 branch / Some SFTP receivers may not receive the full data in the respective SFTP file(s). While validating it with a simple example (SFTP->parallel multicast->2*SFTP in 2 different branches): A test with a large payload showed that both receiver files were created but as soon as one is finished the other one stops too. This already indicates that there are no two different streams that are processed but just one stream that is handed to the SFTP endpoints. Even the trace log showed that if there is cache output stream, the same was used by the SFTP receivers / endpoints in both the branches: *Debug Trace:* 2015 01 13 08:45:21#+00#DEBUG#org.apache.camel.component.file.remote.SftpOperations##anonymous#Camel (Test_Multicast) thread #7 - Multicast##avatarmercury#itjkmerct1#ifl##About to store file: rec2 using stream: *org.apache.camel.converter.stream.FileInputStreamCache@11ef331a|* 2015 01 13 08:45:21#+00#DEBUG#org.apache.camel.component.file.remote.SftpOperations##anonymous#Camel (Test_Multicast) thread #6 - Multicast##avatarmercury#itjkmerct1#ifl##*About to store file: rec1 using stream: org.apache.camel.converter.stream.FileInputStreamCache@11ef331a|* Both use the same stream instance. MulticastProcessor.createProcessorExchangePairs seems to create the copies. The copying is based on Exchange.copy and Message.copy. Message.copy does not copy the streams which is required for the mulicast to work properly with streams. We could circumvent the issue, if we convert the payload / body from cache outputstream to byte[] before the multicast or just before the SFTP endpoints in the multicast branches (i.e. using ${in.bodyAs(byte[])} within camel:simple ). This problem does not come in Sequential multicast. Can you please let us knoiw if this is an issue 9or) already fixed? Thanks, Lakshmi -- View this message in context: http://camel.465427.n5.nabble.com/Data-Corruption-in-SFTP-in-Parallel-Multicast-branches-tp5761673.html Sent from the Camel - Users mailing list archive at Nabble.com.
