Hi Dejan, Currently we have a hard requirement to support large file transfer resume. We're very keen on using ActiveMQ, along with Camel, thus we must implement this "resume" feature. I'm in the investigation stage but I was thinking of implement this the follow way:
Following Lou's suggestion, the producer will break down the large file to be sent into chunks and send them to the broker (ActiveMQ) one at a time until the entire file is sent. In the event of a file transfer interruption, the producer will detect this event and when connection is restore, it will only send the remainder to the broker (with a resume flag or something like that). The broker will then continue to receive the rest of the file knowing that it will have to concatenate the files together before putting it on the queue for consumption. I'm not sure if this is the best solution or not. If you have a better suggestion please feel free to let me know. My personal email address is kevinle92...@yahoo.com Dejan Bosanac wrote: > > Hi, > > just to confirm that ActiveMQ comes with HTTP and FTP strategies for > upload > and download, that doesn't support resume. This can be implemented as a > separate strategy. All contributions are welcomed. > > Cheers > -- > Dejan Bosanac - http://twitter.com/dejanb > > Open Source Integration - http://fusesource.com/ > ActiveMQ in Action - http://www.manning.com/snyder/ > Blog - http://www.nighttale.net > > > On Thu, Jun 10, 2010 at 12:28 AM, kevinle92618 > <kevinle92...@yahoo.com>wrote: > >> >> Thanks Lou. I was afraid of this. If I understand you correctly, >> ActiveMQ >> does NOT support resume transfer. However, you were able to get around >> that >> problem by implementing the snippet codes you mentioned in your previous >> post right? Do you mind posting the codes for the TransferStatus class >> as >> well. Many thanks. >> >> >> Lou Parisi wrote: >> > >> > >> > kevinle92618 wrote: >> >> >> >> I don't have answers to your questions. I'm interested in knowing if >> >> ActiveMQ has the "resume" transfer capability for large file. Please >> >> reply if you have the answer. Thanks. >> >> >> > >> > It has been a while since I worked on this but I did get the "resume" >> > working. As I recall, I kept track of the number of bytes transferred >> and >> > if there was a failure I just had to resume and read past that many >> bytes >> > and restart transfer. Here is a snippet from some sample code I >> created >> > for testing. >> > >> > >> > private static void startTransfer() throws Exception { >> > RandomAccessFile outFile = null; >> > InputStream in = null; >> > byte[] buf = new byte[bufSize]; >> > int len = 0; >> > URLConnection conn = null; >> > >> > // len == -1 when end of InputStream is read >> > while (len != -1) { >> > try { >> > conn = url.openConnection(); >> > conn.setConnectTimeout(connectionTimeout); >> > in = getInputStream(conn); >> > in.skip(transferStatus.getBytesDownloaded()); >> > >> > int count = 0; >> > System.out.println("downloading"); >> > transferStatus.setStatus(TransferStatus.DOWNLOADING); >> > outFile = new RandomAccessFile("build/outfile", "rw"); >> > outFile.seek(transferStatus.getBytesDownloaded()); >> > while ((len = in.read(buf)) > 0) { >> > outFile.write(buf, 0, len); >> > transferStatus.incrementBytesDownloaded(len); >> > updateElapsedTime(); >> > } >> > } catch (Exception e) { >> > System.out.println("Exception transfering data: " + >> > e.getMessage()); >> > } finally { >> > if ( in != null ) { >> > in.close(); >> > } >> > if ( outFile != null ) { >> > outFile.close(); >> > } >> > conn = null; >> > } >> > } >> > System.out.println("transfer complete"); >> > transferStatus.setStatus(TransferStatus.COMPLETE); >> > } >> > >> > >> >> -- >> View this message in context: >> http://old.nabble.com/Large-File-Transfer-tp23378044p28836565.html >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> >> > > > ----- > Dejan Bosanac > > Open Source Integration - http://fusesource.com/ > ActiveMQ in Action - http://www.manning.com/snyder/ > Blog - http://www.nighttale.net > -- View this message in context: http://old.nabble.com/Large-File-Transfer-tp23378044p28843604.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
