Hi Sachith, I would define a thrift struct that has metadata about the file, followed by a member of type list<byte> that stores the file data. I would also define the metadata so it can support transmitting the file in chunks (data that says this is chunk m of n, for instance), where each chunk is sent in a separate message to the server. I would also include some unique cookie value so you can differentiate between two client processes sending the same file.
On the client side, figure out how many chunks you need to send the file, then split the file data up between that many instances of your Thrift class. Iterate over the Thrift classes and send each to the server. On the server side, receive the sendFile messages. Use the unique cookie from within the structure to sort the structures you receive into a separate bin for each file you are receiving (std::map<std::string, std::vector<YourThriftStructType *> > would probably work). When you have received all n chunks for a file (which you can tell because of the metadata), then use the collected Thrift structures to reconstitute the file on the server side. The server method can return a value that tells whether or not it thinks the transmission has completed. This approach lets you field send messages from many clients simultaneously with a single server instance. That's my first cut. It probably requires some refinement. ;-) - Rush On Mar 5, 2014, at 1:45 PM, Sachith Withana wrote: > Thanks Henrique and Jens, > > Ideally we'd like to support multiple Gigabytes of data. > As you said, we will potentially end up having a server to do file > uploading and downloading through URLs for Big files. > > But we'd like to support small file (~10MB) uploads using the Thrift > interface at least for now. > > What would be the best approach in achieving that? > > > On Wed, Mar 5, 2014 at 4:32 PM, Jens Geyer <jensge...@hotmail.com> wrote: > >> [...] faster to send a URL and download it separately, [...] >>> >> >> Ack, I was about to write the same. >> >> >> >> -----Ursprüngliche Nachricht----- From: Henrique Mendonça >> Sent: Wednesday, March 5, 2014 10:24 PM >> To: dev@thrift.apache.org >> Subject: Re: File Upload through thrift >> >> >> Hi, >> You can do this, how big are your files? You might find that some >> implementations do have a size limit, not sure about Java, though. >> Anyway, is generally faster to send a URL and download it separately, if it >> fits you... >> >> >> On 5 March 2014 20:56, Sachith Withana <swsach...@gmail.com> wrote: >> >> I'm planning on converting the file into binary type and then do the >>> transfer to the server. >>> >>> What are the possible drawbacks of using this? >>> >>> This feature is for Apache Airavata. We are dealing with multiple language >>> clients and a main Java Server. >>> We have to upload ( and download) large files too. >>> >>> Thanks. >>> >>> >>> >>> >>> >>> On Wed, Mar 5, 2014 at 1:47 PM, Sachith Withana <swsach...@gmail.com> >>> wrote: >>> >>>> Hi all, >>>> >>>> I need to transfer files between the Thrift clients and Servers. >>>> There can be large files to be transferred. >>>> >>>> Can someone please suggest a way of obtaining this? >>>> >>>> -- >>>> Thanks, >>>> Sachith Withana >>>> >>>> >>> >>> >>> -- >>> Thanks, >>> Sachith Withana >>> >>> >> > > > -- > Thanks, > Sachith Withana
