Hi, I was referring to blob messages as natural way to exchange large files
http://activemq.apache.org/blob-messages.html In that scenario you don't send message payloads to the broker but to the external HTTP or FTP server (those two are currently implemented). You can find the implementation of the current stuff here http://fisheye6.atlassian.com/browse/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/blob and you can try adding needed functionality (or create a new one). <http://activemq.apache.org/blob-messages.html>Cheers -- Dejan Bosanac - http://twitter.com/dejanb Open Source Integration - http://fusesource.com/ ActiveMQ in Action - http://www.manning.com/snyder/ Blog - http://www.nighttale.net On Thu, Jun 10, 2010 at 4:07 PM, kevinle92618 <kevinle92...@yahoo.com>wrote: > > Hi Dejan, > > Currently we have a hard requirement to support large file transfer resume. > We're very keen on using ActiveMQ, along with Camel, thus we must implement > this "resume" feature. I'm in the investigation stage but I was thinking > of > implement this the follow way: > > Following Lou's suggestion, the producer will break down the large file to > be sent into chunks and send them to the broker (ActiveMQ) one at a time > until the entire file is sent. In the event of a file transfer > interruption, the producer will detect this event and when connection is > restore, it will only send the remainder to the broker (with a resume flag > or something like that). The broker will then continue to receive the rest > of the file knowing that it will have to concatenate the files together > before putting it on the queue for consumption. > > I'm not sure if this is the best solution or not. If you have a better > suggestion please feel free to let me know. My personal email address is > kevinle92...@yahoo.com > > > > Dejan Bosanac wrote: > > > > Hi, > > > > just to confirm that ActiveMQ comes with HTTP and FTP strategies for > > upload > > and download, that doesn't support resume. This can be implemented as a > > separate strategy. All contributions are welcomed. > > > > Cheers > > -- > > Dejan Bosanac - http://twitter.com/dejanb > > > > Open Source Integration - http://fusesource.com/ > > ActiveMQ in Action - http://www.manning.com/snyder/ > > Blog - http://www.nighttale.net > > > > > > On Thu, Jun 10, 2010 at 12:28 AM, kevinle92618 > > <kevinle92...@yahoo.com>wrote: > > > >> > >> Thanks Lou. I was afraid of this. If I understand you correctly, > >> ActiveMQ > >> does NOT support resume transfer. However, you were able to get around > >> that > >> problem by implementing the snippet codes you mentioned in your previous > >> post right? Do you mind posting the codes for the TransferStatus class > >> as > >> well. Many thanks. > >> > >> > >> Lou Parisi wrote: > >> > > >> > > >> > kevinle92618 wrote: > >> >> > >> >> I don't have answers to your questions. I'm interested in knowing if > >> >> ActiveMQ has the "resume" transfer capability for large file. Please > >> >> reply if you have the answer. Thanks. > >> >> > >> > > >> > It has been a while since I worked on this but I did get the "resume" > >> > working. As I recall, I kept track of the number of bytes transferred > >> and > >> > if there was a failure I just had to resume and read past that many > >> bytes > >> > and restart transfer. Here is a snippet from some sample code I > >> created > >> > for testing. > >> > > >> > > >> > private static void startTransfer() throws Exception { > >> > RandomAccessFile outFile = null; > >> > InputStream in = null; > >> > byte[] buf = new byte[bufSize]; > >> > int len = 0; > >> > URLConnection conn = null; > >> > > >> > // len == -1 when end of InputStream is read > >> > while (len != -1) { > >> > try { > >> > conn = url.openConnection(); > >> > conn.setConnectTimeout(connectionTimeout); > >> > in = getInputStream(conn); > >> > in.skip(transferStatus.getBytesDownloaded()); > >> > > >> > int count = 0; > >> > System.out.println("downloading"); > >> > transferStatus.setStatus(TransferStatus.DOWNLOADING); > >> > outFile = new RandomAccessFile("build/outfile", "rw"); > >> > outFile.seek(transferStatus.getBytesDownloaded()); > >> > while ((len = in.read(buf)) > 0) { > >> > outFile.write(buf, 0, len); > >> > transferStatus.incrementBytesDownloaded(len); > >> > updateElapsedTime(); > >> > } > >> > } catch (Exception e) { > >> > System.out.println("Exception transfering data: " + > >> > e.getMessage()); > >> > } finally { > >> > if ( in != null ) { > >> > in.close(); > >> > } > >> > if ( outFile != null ) { > >> > outFile.close(); > >> > } > >> > conn = null; > >> > } > >> > } > >> > System.out.println("transfer complete"); > >> > transferStatus.setStatus(TransferStatus.COMPLETE); > >> > } > >> > > >> > > >> > >> -- > >> View this message in context: > >> http://old.nabble.com/Large-File-Transfer-tp23378044p28836565.html > >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. > >> > >> > > > > > > ----- > > Dejan Bosanac > > > > Open Source Integration - http://fusesource.com/ > > ActiveMQ in Action - http://www.manning.com/snyder/ > > Blog - http://www.nighttale.net > > > > -- > View this message in context: > http://old.nabble.com/Large-File-Transfer-tp23378044p28843604.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > >
